Final Data Analysis Project
Write up due April 28 at 5 pm
For this project you will take the role of a consultant hired by a real estate investment firm in Ames, Iowa, a mid-west town in the United States, to analyze data in order to help provide insight into how the firm should invest for highest profits, and to quantify and communicate to the company management what types of real estate properties are good investments and why. They have provided you with data on housing sales from between 2006 to 2010 that contains information about the characteristics of the house (number of bedrooms, number of bathrooms, square footage, etc.) and the house’s sale price. The codebook for this data set is available online here or in the Data folder in your repo.
About the Data Analysis Project
It’s generally a bad idea to buy the most expensive house in the neighborhood. And remember the real estate agents’ mantra: Location, location, location! Keep in mind that the goal is to make money for your investors, and hence investing in a property that is overvalued (costing more than it is worth) is rarely a good idea. This means that it’s critical to know which properties are overvalued and which are undervalued. The company that hired you has many questions for you about the housing market. It is up to you to decide what methods you want to use (frequentist or Bayesian) to answer these questions, and implement them to help to identify undervalued and overvalued properties.
You will have three data sets: a subset for training, a subset for testing, and a third subset for validation. You will be asked to do data exploration and build your model (or models) initially using only the training data. Then, you will test your model on the testing data, and finally validate using the validation data. We are challenging you to keep your analysis experience realistic, and in a realistic scenario you would not have access to all three of these data sets at once. You will be able to see on our scoreboard how well your team is doing based on its predictive performance on the testing data. After your project is turned in you will see the final score on the validation set.
All members of the team should contribute equally and answer any questions about the analysis at the final presentation.
For your analysis create a new notebook named “project.Rmd” and update accordingly rather than editing this.
Read in Training Data
To get started read in the training data.
library(dplyr)
library(tidyr)
load("ames_train.Rdata")
print(paste0("The dataset has ", dim(ames_train)[1], " number of observations and ", dim(ames_train)[2], " features"))
#Variables with NA's and their proportion of missing data
miss = apply(is.na(ames_train), 2, sum)
miss_prop = round(miss[miss>0]/nrow(ames_train), 3)
print(miss_prop)
which(miss_prop>0.5) # four features have greater than 50% of data "missing" -- drop these variablesNotes about data cleaning:
We dropped “utilities” (type of utilities available) since in the training set, only 2 observations did not have all the utilities (electricity, gas, water and sewage). Intuitively, most modern property are equipped with these basic public utilities and keeping the variable would therefore be unnecessary.
We also dropped “condition 2” (proximity to various conditions if more than one is present) since it seemed redundant from our training set. Given Condition 1, only 12 properties were not close to normal conditions.
In the original scale, 1990 and 1900 would not be much different. Therefore, we changed the scale to the number of years since last construction or remodelling, subtracted from year 2010 (the end year in the dataset).
Another variable dropped in our model was “roof material” since only 1% of the property used material other than “standard composite shingle”. Similarly, “heating” was also dropped since more than 95% of the property has gas forced warm air furnace (GasA) instead of other types of heating.
For “exterior quality” and “exterior condition” , we recoded these ordinal variables to 1-5 to replace the original scale of conditions (from poor to excellent). Similarly, we recoded “basement exposure” and “basement rating” except that the new scale would start from 0 for properties without basement.
Exter Qual (Ordinal): Evaluates the quality of the material on the exterior
Ex Excellent
Gd Good
TA Average/Typical
Fa Fair
Po PoorExter Cond (Ordinal): Evaluates the present condition of the material on the exterior
Ex Excellent
Gd Good
TA Average/Typical
Fa Fair
Po PoorContinuous variables such as “1st floor square feet” and “2nd floor square feet” were log-transformed for interpretation purpose.
For variable “functional”, we recoded different ordinal levels into binary levels — typical functionality or not, including minor and major deductions.
We summed up the number of bathrooms to one continuous variable. Note that one half-bathroom would be coded as 0.5.
# Did not remove any NA entries in Lot.frontage
data=ames_train
data <- data %>%
#filter(!is.na(Lot.Frontage)) %>%
mutate(MS.SubClass= factor(MS.SubClass)) %>%
mutate(Alley = factor(Alley, levels = levels(addNA(Alley)), labels = c(levels(Alley), "None"), exclude = NULL)) %>%
mutate(HouseAge = Yr.Sold- pmax(Year.Built, Year.Remod.Add)) %>%
filter(!is.na(Mas.Vnr.Area)) %>%
mutate(Bsmt.YN = 1*(!is.na(Bsmt.Qual))) %>%
mutate(Bsmt.Qual = factor(Bsmt.Qual, levels = levels(addNA(Bsmt.Qual)), labels = c(levels(Bsmt.Qual), "None"), exclude = NULL)) %>%
mutate(Bsmt.Qual = relevel(Bsmt.Qual, ref="None")) %>%
mutate(Bsmt.Cond = factor(Bsmt.Cond, levels = levels(addNA(Bsmt.Cond)), labels = c(levels(Bsmt.Cond), "None"), exclude = NULL)) %>%
mutate(Bsmt.Cond = relevel(Bsmt.Cond, ref="None")) %>%
mutate(Bsmt.Exposure = factor(Bsmt.Exposure, levels = levels(addNA(Bsmt.Exposure)), labels = c(levels(Bsmt.Exposure), "None"), exclude = NULL)) %>%
mutate(Bsmt.Exposure = relevel(Bsmt.Exposure, ref="None")) %>%
mutate(BsmtFin.Type.1= factor(BsmtFin.Type.1, levels = levels(addNA(BsmtFin.Type.1)), labels = c(levels(BsmtFin.Type.1), "None"), exclude = NULL)) %>%
mutate(BsmtFin.Type.1 = relevel(BsmtFin.Type.1, ref="None")) %>%
mutate(BsmtFin.Type.2= factor(BsmtFin.Type.2, levels = levels(addNA(BsmtFin.Type.2)), labels = c(levels(BsmtFin.Type.2), "None"), exclude = NULL)) %>%
mutate(BsmtFin.Type.2 = relevel(BsmtFin.Type.2, ref="None")) %>%
mutate(X12.SF= X1st.Flr.SF+ X2nd.Flr.SF) %>%
filter(!is.na(Bsmt.Full.Bath)) %>%
filter(!is.na(Bsmt.Half.Bath)) %>%
mutate(Baths = Bsmt.Full.Bath + 0.5*Bsmt.Half.Bath + Full.Bath + 0.5*Half.Bath) %>%
mutate(Fireplace.YN = 1*(Fireplaces>0)) %>%
mutate(Fireplace.Qu = factor(Fireplace.Qu, levels = levels(addNA(Fireplace.Qu)), labels = c(levels(Fireplace.Qu), "None"), exclude = NULL)) %>%
mutate(Fireplace.Qu = relevel(Fireplace.Qu, ref="None")) %>%
mutate(Garage.YN = 1*(!is.na(Garage.Cond))) %>%
mutate(Garage.Type = factor(Garage.Type, levels = levels(addNA(Garage.Type)), labels = c(levels(Garage.Type), "None"), exclude = NULL)) %>%
mutate(Garage.Type = relevel(Garage.Type, ref="None")) %>%
mutate(Garage.Finish = factor(Garage.Finish, levels = levels(addNA(Garage.Finish)), labels = c(levels(Garage.Finish), "None"), exclude = NULL)) %>%
mutate(Garage.Finish = relevel(Garage.Finish, ref="None")) %>%
mutate(Garage.Qual = factor(Garage.Qual, levels = levels(addNA(Garage.Qual)), labels = c(levels(Garage.Qual), "None"), exclude = NULL)) %>%
mutate(Garage.Qual = relevel(Garage.Qual, ref="None")) %>%
mutate(Garage.Cond = factor(Garage.Cond, levels = levels(addNA(Garage.Cond)), labels = c(levels(Garage.Cond), "None"), exclude = NULL)) %>%
mutate(Garage.Cond = relevel(Garage.Cond, ref="None")) %>%
mutate(Porch.Area = Wood.Deck.SF+ Open.Porch.SF+Enclosed.Porch+X3Ssn.Porch + Screen.Porch) %>%
mutate(Pool.YN = 1*(Pool.Area>0)) %>%
mutate(Pool.QC = factor(Pool.QC, levels = levels(addNA(Pool.QC)), labels = c(levels(Pool.QC), "None"), exclude = NULL)) %>%
mutate(Pool.QC = relevel(Pool.QC, ref="None")) %>%
mutate(Fence = factor(Fence, levels = levels(addNA(Fence)), labels = c(levels(Fence), "None"), exclude = NULL)) %>%
mutate(Misc.Feature = factor(Misc.Feature, levels = levels(addNA(Misc.Feature)), labels = c(levels(Misc.Feature), "None"), exclude = NULL)) %>%
mutate(Mo.Sold = as.factor(Mo.Sold)) %>%
mutate(Yr.Sold = as.factor(Yr.Sold)) %>%
dplyr::select(-Garage.Yr.Blt) %>%
mutate(Condition.1 = as.character(Condition.1)) %>%
mutate(Kitchen.Qual=plyr::mapvalues(Kitchen.Qual, from = c("Po", "Fa", "TA","Gd", "Ex" ), to = c("1", "2", "3", "4", "5"))) %>%
mutate(Kitchen.Qual = as.numeric(as.character(Kitchen.Qual))) %>%
mutate(Heating.QC=plyr::mapvalues(Heating.QC, from = c("Po", "Fa", "TA","Gd", "Ex" ), to = c("1", "2", "3", "4", "5"))) %>%
mutate(Heating.QC = as.numeric(as.character(Heating.QC))) %>%
mutate(Bsmt.Qual = droplevels(Bsmt.Qual)) %>%
mutate(Functional = droplevels(Functional)) %>%
mutate(Roof.Matl = droplevels(Roof.Matl))
# Simplify Condition 1 (Park, Rail, Normal)
ind_rail<-which(data$Condition.1=="RRNn" | data$Condition.1=="RRAn" | data$Condition.1=="RRNe" | data$Condition.1=="RRAe")
ind_park<-which(data$Condition.1=="PosN" | data$Condition.1=="PosA")
data$Condition.1[ind_rail]<-"Rail"
data$Condition.1[ind_park]<-"Park"
data = data %>%
mutate(Condition.1 = factor(Condition.1)) %>%
mutate(Condition.1 = relevel(Condition.1, ref="Norm"))
# Eliminate the one entry in 'Exposure' that had been left completely empty
data_train<-data
data_train$Bsmt.Exposure[which(data_train$Bsmt.Exposure=="")]<-"None"
data_train$Bsmt.Exposure<-droplevels(data_train$Bsmt.Exposure)
data_train$Pool.Area<-data_train$Pool.Area+1
data_train$Total.Bsmt.SF<-data_train$Total.Bsmt.SF+1The Neighborhood variable, typically of little interest other than to model the location effect, may be of more relevance when used with the map.
We are restricting attention to just the “normal sales” condition.
Part I: Simple Model
In the first model you are allowed only limited manipulations of the original data set to predict the sales price price. You are allowed to take power transformations of the original variables [square roots, logs, inverses, squares, etc.] but you are NOT allowed to create interaction variables. This means that a variable may only be used once in an equation [if you use $ x^2$ don’t use \(x\)]. Additionally, you may eliminate any data points you deem unfit. This model should have a minimum r-square of 73% (in the original units) and contain at least 6 variables but fewer than 20.
### perfromance evlaution function
performance<- function(Y, Yhat){
bias<- mean(Y-Yhat[,1])
max.dev<-max(abs(Y-Yhat[,1]))
mean.dev<-mean(abs(Y-Yhat[,1]))
RMSE<-sqrt(mean((Y-Yhat[,1])^2))
coverage<-mean((Y>Yhat[,2]) & (Y<Yhat[,3]))
out<-data.frame(bias=bias, max.dev=max.dev, mean.dev=mean.dev, RMES=RMSE, Coverage=coverage)
return(out)
}library(MASS)
# Base model with transformed predictors
model=lm(price ~ MS.SubClass + MS.Zoning + log(Lot.Frontage) + log(Lot.Area) + Street + Alley + Lot.Shape + Land.Contour + Lot.Config + Land.Slope + Neighborhood + Condition.1 + Bldg.Type + House.Style + Overall.Qual + Overall.Cond + HouseAge + Roof.Style + Roof.Matl + Exterior.1st + Mas.Vnr.Type + log(1+Mas.Vnr.Area) + Exter.Cond + Exter.Qual + Foundation + Bsmt.Qual + Bsmt.Cond + Bsmt.Exposure + Total.Bsmt.SF + Heating + Heating.QC + Central.Air + Electrical + log(X12.SF) + log(1+Low.Qual.Fin.SF) + Baths + Bedroom.AbvGr + Kitchen.AbvGr + Kitchen.Qual + Functional + Fireplaces + Fireplace.Qu + Garage.Type + Garage.Finish + Garage.Cars + Garage.Cond + Garage.Qual + Paved.Drive + log(1+Pool.Area) + Pool.QC + Fence + Misc.Val + Mo.Sold +Yr.Sold + Sale.Type + TotalSq, data=data_train)
# Boxcox (indicates that log is decent)
l<-boxcox(model)
expo<-round(l$x[which.max(l$y)],2)
## Current model
## log(Lot.Frontage) currently removed to have more data points (was not included when left in the model with BIC)
model.0=lm(log(price) ~ MS.SubClass + MS.Zoning + log(Lot.Area) + Street + Alley + Lot.Shape + Land.Contour + Lot.Config + Land.Slope + Neighborhood + Condition.1 + Bldg.Type + House.Style + Overall.Qual + Overall.Cond + HouseAge + Roof.Style + Roof.Matl + Exterior.1st + Mas.Vnr.Type + log(1+Mas.Vnr.Area) + Exter.Cond + Exter.Qual + Foundation + Bsmt.Qual + Bsmt.Cond + Bsmt.Exposure + log(Total.Bsmt.SF) + Bsmt.YN+ Heating + Heating.QC + Central.Air + log(1+Low.Qual.Fin.SF) + Baths + Bedroom.AbvGr + Kitchen.AbvGr + Kitchen.Qual + Functional + Fireplaces + Fireplace.Qu + Garage.Type + Garage.Finish + Garage.Cars +Garage.Qual+Garage.Cond + Paved.Drive + log(Pool.Area) + Pool.QC + Fence + Misc.Val + Mo.Sold +Yr.Sold + Sale.Type + log(TotalSq) + Pool.YN , data=data_train)
# There are some perfect collinearities in this model -> eliminate it via AIC/BIC
summary(model.0)
Call:
lm(formula = log(price) ~ MS.SubClass + MS.Zoning + log(Lot.Area) +
Street + Alley + Lot.Shape + Land.Contour + Lot.Config +
Land.Slope + Neighborhood + Condition.1 + Bldg.Type + House.Style +
Overall.Qual + Overall.Cond + HouseAge + Roof.Style + Roof.Matl +
Exterior.1st + Mas.Vnr.Type + log(1 + Mas.Vnr.Area) + Exter.Cond +
Exter.Qual + Foundation + Bsmt.Qual + Bsmt.Cond + Bsmt.Exposure +
log(Total.Bsmt.SF) + Bsmt.YN + Heating + Heating.QC + Central.Air +
log(1 + Low.Qual.Fin.SF) + Baths + Bedroom.AbvGr + Kitchen.AbvGr +
Kitchen.Qual + Functional + Fireplaces + Fireplace.Qu + Garage.Type +
Garage.Finish + Garage.Cars + Garage.Qual + Garage.Cond +
Paved.Drive + log(Pool.Area) + Pool.QC + Fence + Misc.Val +
Mo.Sold + Yr.Sold + Sale.Type + log(TotalSq) + Pool.YN, data = data_train)
Residuals:
Min 1Q Median 3Q Max
-0.43874 -0.04567 0.00024 0.04925 0.25557
Coefficients: (8 not defined because of singularities)
Estimate Std. Error t value Pr(>|t|)
(Intercept) 7.135e+00 2.287e-01 31.199 < 2e-16 ***
MS.SubClass30 -7.720e-02 1.621e-02 -4.762 2.13e-06 ***
MS.SubClass40 -2.625e-02 5.441e-02 -0.483 0.629528
MS.SubClass45 -1.585e-01 1.216e-01 -1.304 0.192564
MS.SubClass50 -2.953e-02 3.292e-02 -0.897 0.369868
MS.SubClass60 -3.433e-02 2.747e-02 -1.250 0.211688
MS.SubClass70 -7.417e-02 2.987e-02 -2.483 0.013157 *
MS.SubClass75 -1.948e-02 8.094e-02 -0.241 0.809826
MS.SubClass80 -6.554e-02 4.932e-02 -1.329 0.184105
MS.SubClass85 -3.716e-02 3.572e-02 -1.040 0.298403
MS.SubClass90 -2.750e-02 2.873e-02 -0.957 0.338788
MS.SubClass120 3.104e-02 4.763e-02 0.652 0.514768
MS.SubClass150 -1.220e-01 1.210e-01 -1.008 0.313770
MS.SubClass160 -5.059e-02 5.922e-02 -0.854 0.393087
MS.SubClass180 -2.540e-02 8.118e-02 -0.313 0.754415
MS.SubClass190 -7.996e-02 9.993e-02 -0.800 0.423747
MS.ZoningC (all) -8.623e-02 1.100e-01 -0.784 0.433200
MS.ZoningFV 1.407e-01 1.042e-01 1.351 0.176927
MS.ZoningI (all) -2.767e-02 1.291e-01 -0.214 0.830289
MS.ZoningRH 1.659e-01 1.037e-01 1.599 0.110075
MS.ZoningRL 1.231e-01 9.957e-02 1.236 0.216735
MS.ZoningRM 7.809e-02 1.008e-01 0.775 0.438763
log(Lot.Area) 9.002e-02 1.010e-02 8.912 < 2e-16 ***
StreetPave -1.815e-02 4.429e-02 -0.410 0.682128
AlleyPave -1.134e-02 2.326e-02 -0.488 0.625946
AlleyNone 1.801e-02 1.395e-02 1.292 0.196747
Lot.ShapeIR2 -6.732e-04 1.582e-02 -0.043 0.966069
Lot.ShapeIR3 6.258e-03 3.092e-02 0.202 0.839656
Lot.ShapeReg -1.546e-03 5.964e-03 -0.259 0.795449
Land.ContourHLS 2.953e-02 1.852e-02 1.594 0.111086
Land.ContourLow -5.651e-03 2.377e-02 -0.238 0.812139
Land.ContourLvl 2.114e-02 1.375e-02 1.537 0.124571
Lot.ConfigCulDSac 1.419e-02 1.163e-02 1.220 0.222861
Lot.ConfigFR2 -3.046e-02 1.445e-02 -2.108 0.035210 *
Lot.ConfigFR3 -2.514e-02 4.191e-02 -0.600 0.548733
Lot.ConfigInside 1.273e-02 6.572e-03 1.937 0.053009 .
Land.SlopeMod 1.605e-02 1.453e-02 1.104 0.269592
Land.SlopeSev -7.217e-02 5.346e-02 -1.350 0.177255
NeighborhoodBlueste 1.012e-01 4.633e-02 2.184 0.029154 *
NeighborhoodBrDale 1.193e-02 4.322e-02 0.276 0.782511
NeighborhoodBrkSide 2.929e-02 3.587e-02 0.817 0.414264
NeighborhoodClearCr 1.518e-02 3.742e-02 0.406 0.685092
NeighborhoodCollgCr -4.175e-03 3.026e-02 -0.138 0.890275
NeighborhoodCrawfor 6.731e-02 3.434e-02 1.960 0.050179 .
NeighborhoodEdwards -8.750e-02 3.226e-02 -2.712 0.006778 **
NeighborhoodGilbert -4.620e-02 3.155e-02 -1.464 0.143425
NeighborhoodGreens 5.330e-02 4.491e-02 1.187 0.235555
NeighborhoodGrnHill 4.587e-01 7.033e-02 6.522 9.94e-11 ***
NeighborhoodIDOTRR -4.480e-02 3.976e-02 -1.127 0.260077
NeighborhoodLandmrk -8.096e-02 9.648e-02 -0.839 0.401562
NeighborhoodMeadowV -9.830e-02 4.989e-02 -1.970 0.049004 *
NeighborhoodMitchel -1.939e-02 3.247e-02 -0.597 0.550505
NeighborhoodNAmes -5.754e-02 3.165e-02 -1.818 0.069271 .
NeighborhoodNoRidge 7.880e-02 3.380e-02 2.332 0.019881 *
NeighborhoodNPkVill 4.515e-02 4.172e-02 1.082 0.279425
NeighborhoodNridgHt 3.993e-02 3.232e-02 1.235 0.216929
NeighborhoodNWAmes -3.705e-02 3.264e-02 -1.135 0.256547
NeighborhoodOldTown -5.117e-02 3.647e-02 -1.403 0.160835
NeighborhoodSawyer -1.697e-02 3.244e-02 -0.523 0.600886
NeighborhoodSawyerW -3.472e-02 3.147e-02 -1.103 0.270103
NeighborhoodSomerst 7.220e-02 4.009e-02 1.801 0.071939 .
NeighborhoodStoneBr 7.764e-02 3.451e-02 2.250 0.024620 *
NeighborhoodSWISU -4.571e-02 3.748e-02 -1.220 0.222869
NeighborhoodTimber -2.846e-02 3.383e-02 -0.841 0.400336
NeighborhoodVeenker 2.246e-02 3.973e-02 0.565 0.571929
Condition.1Artery -7.044e-02 1.535e-02 -4.589 4.90e-06 ***
Condition.1Feedr -7.402e-02 1.141e-02 -6.486 1.25e-10 ***
Condition.1Park -2.670e-04 1.739e-02 -0.015 0.987752
Condition.1Rail -5.997e-02 1.480e-02 -4.052 5.38e-05 ***
Bldg.Type2fmCon 3.790e-02 9.699e-02 0.391 0.696031
Bldg.TypeDuplex NA NA NA NA
Bldg.TypeTwnhs -4.101e-02 4.983e-02 -0.823 0.410662
Bldg.TypeTwnhsE -3.401e-02 4.657e-02 -0.730 0.465340
House.Style1.5Unf 1.452e-01 1.200e-01 1.210 0.226482
House.Style1Story 4.451e-02 3.137e-02 1.419 0.156180
House.Style2.5Fin -8.497e-02 9.804e-02 -0.867 0.386254
House.Style2.5Unf 4.834e-03 8.411e-02 0.057 0.954181
House.Style2Story 4.421e-02 2.947e-02 1.500 0.133822
House.StyleSFoyer 9.402e-02 4.126e-02 2.279 0.022844 *
House.StyleSLvl 1.099e-01 5.280e-02 2.082 0.037563 *
Overall.Qual 4.386e-02 3.647e-03 12.025 < 2e-16 ***
Overall.Cond 3.438e-02 3.001e-03 11.455 < 2e-16 ***
HouseAge -5.985e-04 1.928e-04 -3.104 0.001954 **
Roof.StyleGable -6.870e-02 5.100e-02 -1.347 0.178219
Roof.StyleGambrel -1.341e-01 5.823e-02 -2.303 0.021454 *
Roof.StyleHip -7.569e-02 5.137e-02 -1.473 0.140909
Roof.StyleMansard -9.772e-02 6.256e-02 -1.562 0.118568
Roof.StyleShed -1.860e-02 9.228e-02 -0.202 0.840325
Roof.MatlMembran 8.715e-02 1.139e-01 0.765 0.444332
Roof.MatlRoll 7.002e-02 9.408e-02 0.744 0.456888
Roof.MatlTar&Grv 3.100e-02 3.754e-02 0.826 0.408998
Roof.MatlWdShake 1.155e-02 4.850e-02 0.238 0.811768
Roof.MatlWdShngl 7.904e-02 5.041e-02 1.568 0.117148
Exterior.1stAsphShn -3.774e-02 9.304e-02 -0.406 0.685059
Exterior.1stBrkComm 1.197e-01 7.003e-02 1.710 0.087573 .
Exterior.1stBrkFace 6.513e-02 2.716e-02 2.398 0.016616 *
Exterior.1stCBlock NA NA NA NA
Exterior.1stCemntBd 4.768e-02 2.816e-02 1.693 0.090731 .
Exterior.1stHdBoard 1.592e-02 2.409e-02 0.661 0.508939
Exterior.1stImStucc -1.087e-02 9.163e-02 -0.119 0.905605
Exterior.1stMetalSd 2.877e-02 2.355e-02 1.222 0.222118
Exterior.1stPlywood 9.545e-03 2.532e-02 0.377 0.706277
Exterior.1stPreCast 3.289e-01 1.013e-01 3.245 0.001204 **
Exterior.1stStucco 1.632e-02 3.006e-02 0.543 0.587327
Exterior.1stVinylSd 3.341e-02 2.395e-02 1.395 0.163211
Exterior.1stWd Sdng 1.428e-02 2.354e-02 0.607 0.544233
Exterior.1stWdShing 2.649e-02 2.844e-02 0.932 0.351759
Mas.Vnr.TypeBrkFace 5.390e-02 2.675e-02 2.015 0.044108 *
Mas.Vnr.TypeNone 1.103e-01 3.638e-02 3.031 0.002488 **
Mas.Vnr.TypeStone 5.822e-02 2.849e-02 2.044 0.041189 *
log(1 + Mas.Vnr.Area) 1.221e-02 5.113e-03 2.387 0.017131 *
Exter.CondFa -2.636e-02 4.346e-02 -0.606 0.544314
Exter.CondGd 2.375e-02 3.777e-02 0.629 0.529609
Exter.CondPo -1.132e-01 1.012e-01 -1.119 0.263270
Exter.CondTA 3.107e-02 3.777e-02 0.823 0.410846
Exter.QualFa -5.306e-02 3.688e-02 -1.439 0.150478
Exter.QualGd -5.349e-02 1.970e-02 -2.715 0.006716 **
Exter.QualTA -5.260e-02 2.179e-02 -2.414 0.015904 *
FoundationCBlock 2.047e-02 1.118e-02 1.831 0.067407 .
FoundationPConc 4.603e-02 1.188e-02 3.873 0.000113 ***
FoundationSlab -6.749e-03 3.072e-02 -0.220 0.826142
FoundationStone 4.719e-02 4.164e-02 1.133 0.257252
FoundationWood 5.419e-02 5.410e-02 1.002 0.316657
Bsmt.QualEx -5.934e-01 1.230e-01 -4.826 1.56e-06 ***
Bsmt.QualFa -6.228e-01 1.225e-01 -5.084 4.25e-07 ***
Bsmt.QualGd -6.587e-01 1.224e-01 -5.383 8.69e-08 ***
Bsmt.QualPo -1.649e-01 1.733e-01 -0.952 0.341390
Bsmt.QualTA -6.586e-01 1.221e-01 -5.395 8.14e-08 ***
Bsmt.CondEx -5.374e-04 6.234e-02 -0.009 0.993124
Bsmt.CondFa -2.399e-02 1.472e-02 -1.630 0.103428
Bsmt.CondGd 1.951e-03 1.303e-02 0.150 0.881017
Bsmt.CondPo 8.582e-02 7.715e-02 1.112 0.266197
Bsmt.CondTA NA NA NA NA
Bsmt.ExposureAv 1.326e-01 8.588e-02 1.544 0.122909
Bsmt.ExposureGd 1.633e-01 8.631e-02 1.892 0.058769 .
Bsmt.ExposureMn 9.901e-02 8.612e-02 1.150 0.250533
Bsmt.ExposureNo 1.089e-01 8.585e-02 1.269 0.204848
log(Total.Bsmt.SF) 9.271e-02 1.217e-02 7.616 5.06e-14 ***
Bsmt.YN NA NA NA NA
HeatingGasA 7.688e-02 9.347e-02 0.823 0.410924
HeatingGasW 1.523e-01 9.678e-02 1.574 0.115838
HeatingGrav 1.256e-02 1.146e-01 0.110 0.912736
HeatingOthW 3.011e-02 1.146e-01 0.263 0.792809
HeatingWall 1.054e-01 1.038e-01 1.015 0.310188
Heating.QC 8.207e-03 3.392e-03 2.419 0.015689 *
Central.AirY 5.164e-02 1.337e-02 3.862 0.000118 ***
log(1 + Low.Qual.Fin.SF) 1.195e-02 4.013e-03 2.977 0.002968 **
Baths 4.982e-02 4.577e-03 10.884 < 2e-16 ***
Bedroom.AbvGr -1.038e-02 4.588e-03 -2.263 0.023781 *
Kitchen.AbvGr -1.030e-01 2.454e-02 -4.199 2.87e-05 ***
Kitchen.Qual 3.426e-02 5.840e-03 5.866 5.66e-09 ***
FunctionalMaj2 -2.494e-01 5.193e-02 -4.804 1.74e-06 ***
FunctionalMin1 -1.625e-02 3.566e-02 -0.456 0.648641
FunctionalMin2 -1.274e-02 3.606e-02 -0.353 0.723946
FunctionalMod -3.512e-02 3.942e-02 -0.891 0.373152
FunctionalTyp 4.746e-02 3.290e-02 1.443 0.149399
Fireplaces 2.516e-02 8.781e-03 2.865 0.004240 **
Fireplace.QuEx 1.185e-02 2.345e-02 0.505 0.613479
Fireplace.QuFa -3.809e-03 1.740e-02 -0.219 0.826753
Fireplace.QuGd 8.029e-03 1.221e-02 0.658 0.510907
Fireplace.QuPo 1.114e-02 2.043e-02 0.545 0.585585
Fireplace.QuTA 1.034e-03 1.211e-02 0.085 0.931945
Garage.Type2Types -2.409e-02 3.354e-02 -0.718 0.472872
Garage.TypeAttchd 7.027e-03 1.677e-02 0.419 0.675292
Garage.TypeBasment -1.302e-02 3.140e-02 -0.415 0.678518
Garage.TypeBuiltIn 1.970e-02 1.991e-02 0.989 0.322625
Garage.TypeCarPort -7.919e-02 5.498e-02 -1.440 0.150039
Garage.TypeDetchd 1.634e-02 1.637e-02 0.998 0.318665
Garage.Finish -7.485e-02 1.519e-01 -0.493 0.622210
Garage.FinishFin 1.826e-02 8.378e-03 2.180 0.029443 *
Garage.FinishRFn -5.550e-03 7.428e-03 -0.747 0.455052
Garage.FinishUnf NA NA NA NA
Garage.Cars 3.427e-02 5.610e-03 6.109 1.33e-09 ***
Garage.QualFa -1.092e-02 1.439e-02 -0.759 0.448201
Garage.QualGd 4.510e-02 2.910e-02 1.550 0.121430
Garage.QualPo -3.117e-01 7.767e-02 -4.013 6.35e-05 ***
Garage.QualTA NA NA NA NA
Garage.CondFa -6.362e-02 1.804e-02 -3.527 0.000435 ***
Garage.CondGd 6.926e-04 4.160e-02 0.017 0.986719
Garage.CondPo 7.258e-02 4.399e-02 1.650 0.099151 .
Garage.CondTA NA NA NA NA
Paved.DriveP -1.649e-02 1.865e-02 -0.884 0.376870
Paved.DriveY 2.848e-02 1.229e-02 2.318 0.020627 *
log(Pool.Area) -2.200e-01 2.224e-01 -0.989 0.322688
Pool.QCEx 1.317e+00 1.152e+00 1.143 0.253103
Pool.QCFa 1.404e+00 1.416e+00 0.991 0.321709
Pool.QCGd 1.446e+00 1.447e+00 0.999 0.317933
Pool.QCTA 1.684e+00 1.362e+00 1.236 0.216777
FenceGdWo 1.227e-03 1.707e-02 0.072 0.942711
FenceMnPrv -1.599e-02 1.353e-02 -1.182 0.237505
FenceMnWw -4.282e-02 3.570e-02 -1.199 0.230655
FenceNone -1.278e-02 1.234e-02 -1.036 0.300596
Misc.Val 8.128e-07 4.966e-06 0.164 0.870024
Mo.Sold2 -9.601e-03 1.705e-02 -0.563 0.573370
Mo.Sold3 -1.316e-02 1.525e-02 -0.863 0.388463
Mo.Sold4 1.720e-02 1.479e-02 1.163 0.245173
Mo.Sold5 1.143e-02 1.415e-02 0.808 0.419406
Mo.Sold6 1.087e-02 1.386e-02 0.785 0.432847
Mo.Sold7 1.317e-02 1.400e-02 0.941 0.347126
Mo.Sold8 1.980e-03 1.537e-02 0.129 0.897511
[ reached getOption("max.print") -- omitted 18 rows ]
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 0.08396 on 1283 degrees of freedom
Multiple R-squared: 0.9562, Adjusted R-squared: 0.9491
F-statistic: 134 on 209 and 1283 DF, p-value: < 2.2e-16plot(model.0)not plotting observations with leverage one:
47, 168, 242, 284, 411, 638, 655, 739, 804, 990, 1011, 1194, 1198, 1324, 1344, 1376
not plotting observations with leverage one:
47, 168, 242, 284, 411, 638, 655, 739, 804, 990, 1011, 1194, 1198, 1324, 1344, 1376
NaNs producedNaNs produced
#AIC
#model.AIC=step(model.0, k=2)
#summary(model.AIC)
#plot(model.AIC)
#BIC
#model.BIC=step(model.0, k=log(nrow(data_train)))
model.BIC=lm(formula = log(price) ~ log(Lot.Area) + Neighborhood + Condition.1 +
Overall.Qual + Overall.Cond + HouseAge + Foundation + Bsmt.Qual +
Bsmt.Exposure + log(Total.Bsmt.SF) + Heating.QC + Central.Air +
Baths + Bedroom.AbvGr + Kitchen.AbvGr + Kitchen.Qual + Functional +
Fireplaces + Garage.Cars + Paved.Drive + log(Pool.Area) +
log(TotalSq), data = data_train)
summary(model.BIC)
Call:
lm(formula = log(price) ~ log(Lot.Area) + Neighborhood + Condition.1 +
Overall.Qual + Overall.Cond + HouseAge + Foundation + Bsmt.Qual +
Bsmt.Exposure + log(Total.Bsmt.SF) + Heating.QC + Central.Air +
Baths + Bedroom.AbvGr + Kitchen.AbvGr + Kitchen.Qual + Functional +
Fireplaces + Garage.Cars + Paved.Drive + log(Pool.Area) +
log(TotalSq), data = data_train)
Residuals:
Min 1Q Median 3Q Max
-0.66831 -0.05150 0.00025 0.05629 0.32251
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 7.3926554 0.1169000 63.239 < 2e-16 ***
log(Lot.Area) 0.1026436 0.0075547 13.587 < 2e-16 ***
NeighborhoodBlueste 0.0146631 0.0419500 0.350 0.726738
NeighborhoodBrDale -0.0634618 0.0347412 -1.827 0.067954 .
NeighborhoodBrkSide -0.0219612 0.0306986 -0.715 0.474491
NeighborhoodClearCr 0.0028061 0.0338084 0.083 0.933862
NeighborhoodCollgCr -0.0131967 0.0275606 -0.479 0.632136
NeighborhoodCrawfor 0.0479177 0.0311068 1.540 0.123679
NeighborhoodEdwards -0.1000400 0.0294309 -3.399 0.000695 ***
NeighborhoodGilbert -0.0379098 0.0287188 -1.320 0.187036
NeighborhoodGreens 0.0359676 0.0442891 0.812 0.416864
NeighborhoodGrnHill 0.4397167 0.0709210 6.200 7.38e-10 ***
NeighborhoodIDOTRR -0.1451682 0.0324860 -4.469 8.49e-06 ***
NeighborhoodLandmrk -0.0681952 0.0949941 -0.718 0.472943
NeighborhoodMeadowV -0.1266187 0.0383000 -3.306 0.000970 ***
NeighborhoodMitchel -0.0368471 0.0295088 -1.249 0.211987
NeighborhoodNAmes -0.0565614 0.0285542 -1.981 0.047801 *
NeighborhoodNoRidge 0.0623060 0.0303150 2.055 0.040034 *
NeighborhoodNPkVill -0.0208120 0.0384833 -0.541 0.588726
NeighborhoodNridgHt 0.0509080 0.0290237 1.754 0.079643 .
NeighborhoodNWAmes -0.0544560 0.0297828 -1.828 0.067693 .
NeighborhoodOldTown -0.1188641 0.0294370 -4.038 5.68e-05 ***
NeighborhoodSawyer -0.0229563 0.0298927 -0.768 0.442639
NeighborhoodSawyerW -0.0515292 0.0289758 -1.778 0.075559 .
NeighborhoodSomerst 0.0566938 0.0276847 2.048 0.040759 *
NeighborhoodStoneBr 0.0745779 0.0321385 2.321 0.020453 *
NeighborhoodSWISU -0.0604508 0.0340779 -1.774 0.076293 .
NeighborhoodTimber -0.0324647 0.0317804 -1.022 0.307176
NeighborhoodVeenker -0.0167331 0.0376356 -0.445 0.656671
Condition.1Artery -0.0760394 0.0151294 -5.026 5.64e-07 ***
Condition.1Feedr -0.0767356 0.0113456 -6.763 1.96e-11 ***
Condition.1Park 0.0069725 0.0182004 0.383 0.701706
Condition.1Rail -0.0496206 0.0149453 -3.320 0.000922 ***
Overall.Qual 0.0494652 0.0034280 14.430 < 2e-16 ***
Overall.Cond 0.0351455 0.0027881 12.605 < 2e-16 ***
HouseAge -0.0008503 0.0001883 -4.516 6.84e-06 ***
FoundationCBlock 0.0583435 0.0105020 5.555 3.30e-08 ***
FoundationPConc 0.0718433 0.0115567 6.217 6.66e-10 ***
FoundationSlab 0.0630207 0.0291017 2.166 0.030512 *
FoundationStone 0.0038723 0.0390916 0.099 0.921107
FoundationWood 0.0465776 0.0554929 0.839 0.401418
Bsmt.QualEx -0.7570306 0.1136937 -6.659 3.94e-11 ***
Bsmt.QualFa -0.8357544 0.1124732 -7.431 1.85e-13 ***
Bsmt.QualGd -0.8407558 0.1123115 -7.486 1.24e-13 ***
Bsmt.QualPo -0.8411315 0.1443930 -5.825 7.04e-09 ***
Bsmt.QualTA -0.8503101 0.1122157 -7.577 6.31e-14 ***
Bsmt.ExposureAv 0.1590409 0.0918543 1.731 0.083588 .
Bsmt.ExposureGd 0.1929930 0.0921545 2.094 0.036415 *
Bsmt.ExposureMn 0.1151061 0.0920620 1.250 0.211392
Bsmt.ExposureNo 0.1296663 0.0917945 1.413 0.158000
log(Total.Bsmt.SF) 0.1191674 0.0091824 12.978 < 2e-16 ***
Heating.QC 0.0091548 0.0033638 2.722 0.006577 **
Central.AirY 0.0587416 0.0121548 4.833 1.49e-06 ***
Baths 0.0470497 0.0045251 10.397 < 2e-16 ***
Bedroom.AbvGr -0.0140992 0.0042615 -3.308 0.000961 ***
Kitchen.AbvGr -0.0977644 0.0140392 -6.964 5.05e-12 ***
Kitchen.Qual 0.0346078 0.0056027 6.177 8.51e-10 ***
FunctionalMaj2 -0.1661268 0.0518564 -3.204 0.001387 **
FunctionalMin1 0.0143539 0.0345741 0.415 0.678085
FunctionalMin2 0.0352368 0.0342227 1.030 0.303356
FunctionalMod -0.0059146 0.0372751 -0.159 0.873948
FunctionalTyp 0.0882264 0.0312921 2.819 0.004877 **
Fireplaces 0.0278606 0.0047452 5.871 5.37e-09 ***
Garage.Cars 0.0400643 0.0046975 8.529 < 2e-16 ***
Paved.DriveP 0.0101521 0.0181899 0.558 0.576854
Paved.DriveY 0.0573398 0.0114111 5.025 5.67e-07 ***
log(Pool.Area) 0.0175609 0.0059007 2.976 0.002969 **
log(TotalSq) 0.3701184 0.0154033 24.029 < 2e-16 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 0.09123 on 1425 degrees of freedom
Multiple R-squared: 0.9426, Adjusted R-squared: 0.9399
F-statistic: 349 on 67 and 1425 DF, p-value: < 2.2e-16plot(model.BIC)not plotting observations with leverage one:
638, 655, 990
not plotting observations with leverage one:
638, 655, 990
# Exploring the remaining predictors relationship to price
# plot(log(data_train$Lot.Area), log(data_train$price))
# plot((data_train$Neighborhood), log(data_train$price))
# plot((data_train$Condition.1), log(data_train$price))
# plot((data_train$Overall.Qual), log(data_train$price))
# plot((data_train$Overall.Cond), log(data_train$price))
# plot((data_train$HouseAge), log(data_train$price))
# plot((data_train$Bsmt.Qual), log(data_train$price))
# plot((data_train$Bsmt.Exposure), log(data_train$price))
# plot(log(data_train$Total.Bsmt.SF), log(data_train$price))
# plot((data_train$Heating.QC), log(data_train$price))
# plot((data_train$Central.Air), log(data_train$price))
# plot((data_train$Baths), log(data_train$price))
# plot((data_train$Bedroom.AbvGr), log(data_train$price))
# plot((data_train$Kitchen.AbvGr), log(data_train$price))
# plot((data_train$Kitchen.Qual), log(data_train$price))
# plot((data_train$Functional), log(data_train$price))
# plot((data_train$Fireplaces), log(data_train$price))
# plot((data_train$Paved.Drive), log(data_train$price))
# plot((data_train$Garage.Cars), log(data_train$price))
# plot((data_train$Garage.Cars), log(data_train$price))
# plot(log(1+data_train$Pool.Area), log(data_train$price))
# plot(log(data_train$TotalSq), log(data_train$price))
termplot(model.BIC, partial.resid = TRUE, col.res = "purple", cex = 0.5,
rug = T, se = T, smooth = panel.smooth)
# There are 3 high leverage points - may want to exclude them
hh<-hatvalues(model.BIC)
id<-which(hh==1)
plot(hatvalues(model.BIC), type = "h")
## Prepare test data
load("ames_test.Rdata")
data=ames_test
data <- data %>%
#filter(!is.na(Lot.Frontage)) %>%
mutate(MS.SubClass= factor(MS.SubClass)) %>%
mutate(Alley = factor(Alley, levels = levels(addNA(Alley)), labels = c(levels(Alley), "None"), exclude = NULL)) %>%
mutate(HouseAge = Yr.Sold- pmax(Year.Built, Year.Remod.Add)) %>%
#filter(!is.na(Mas.Vnr.Area)) %>%
mutate(Bsmt.YN = 1*(!is.na(Bsmt.Qual))) %>%
mutate(Bsmt.Qual = factor(Bsmt.Qual, levels = levels(addNA(Bsmt.Qual)), labels = c(levels(Bsmt.Qual), "None"), exclude = NULL)) %>%
mutate(Bsmt.Qual = relevel(Bsmt.Qual, ref="None")) %>%
mutate(Bsmt.Cond = factor(Bsmt.Cond, levels = levels(addNA(Bsmt.Cond)), labels = c(levels(Bsmt.Cond), "None"), exclude = NULL)) %>%
mutate(Bsmt.Cond = relevel(Bsmt.Cond, ref="None")) %>%
mutate(Bsmt.Exposure = factor(Bsmt.Exposure, levels = levels(addNA(Bsmt.Exposure)), labels = c(levels(Bsmt.Exposure), "None"), exclude = NULL)) %>%
mutate(Bsmt.Exposure = relevel(Bsmt.Exposure, ref="None")) %>%
mutate(BsmtFin.Type.1= factor(BsmtFin.Type.1, levels = levels(addNA(BsmtFin.Type.1)), labels = c(levels(BsmtFin.Type.1), "None"), exclude = NULL)) %>%
mutate(BsmtFin.Type.1 = relevel(BsmtFin.Type.1, ref="None")) %>%
mutate(BsmtFin.Type.2= factor(BsmtFin.Type.2, levels = levels(addNA(BsmtFin.Type.2)), labels = c(levels(BsmtFin.Type.2), "None"), exclude = NULL)) %>%
mutate(BsmtFin.Type.2 = relevel(BsmtFin.Type.2, ref="None")) %>%
mutate(X12.SF= X1st.Flr.SF+ X2nd.Flr.SF) %>%
#filter(!is.na(Bsmt.Full.Bath)) %>%
#filter(!is.na(Bsmt.Half.Bath)) %>%
mutate(Baths = Bsmt.Full.Bath + 0.5*Bsmt.Half.Bath + Full.Bath + 0.5*Half.Bath) %>%
mutate(Fireplace.YN = 1*(Fireplaces>0)) %>%
mutate(Fireplace.Qu = factor(Fireplace.Qu, levels = levels(addNA(Fireplace.Qu)), labels = c(levels(Fireplace.Qu), "None"), exclude = NULL)) %>%
mutate(Fireplace.Qu = relevel(Fireplace.Qu, ref="None")) %>%
mutate(Garage.YN = 1*(!is.na(Garage.Cond))) %>%
mutate(Garage.Type = factor(Garage.Type, levels = levels(addNA(Garage.Type)), labels = c(levels(Garage.Type), "None"), exclude = NULL)) %>%
mutate(Garage.Type = relevel(Garage.Type, ref="None")) %>%
mutate(Garage.Finish = factor(Garage.Finish, levels = levels(addNA(Garage.Finish)), labels = c(levels(Garage.Finish), "None"), exclude = NULL)) %>%
mutate(Garage.Finish = relevel(Garage.Finish, ref="None")) %>%
mutate(Garage.Qual = factor(Garage.Qual, levels = levels(addNA(Garage.Qual)), labels = c(levels(Garage.Qual), "None"), exclude = NULL)) %>%
mutate(Garage.Qual = relevel(Garage.Qual, ref="None")) %>%
mutate(Garage.Cond = factor(Garage.Cond, levels = levels(addNA(Garage.Cond)), labels = c(levels(Garage.Cond), "None"), exclude = NULL)) %>%
mutate(Garage.Cond = relevel(Garage.Cond, ref="None")) %>%
mutate(Porch.Area = Wood.Deck.SF+ Open.Porch.SF+Enclosed.Porch+X3Ssn.Porch + Screen.Porch) %>%
mutate(Pool.YN = 1*(Pool.Area>0)) %>%
mutate(Pool.QC = factor(Pool.QC, levels = levels(addNA(Pool.QC)), labels = c(levels(Pool.QC), "None"), exclude = NULL)) %>%
mutate(Pool.QC = relevel(Pool.QC, ref="None")) %>%
mutate(Fence = factor(Fence, levels = levels(addNA(Fence)), labels = c(levels(Fence), "None"), exclude = NULL)) %>%
mutate(Misc.Feature = factor(Misc.Feature, levels = levels(addNA(Misc.Feature)), labels = c(levels(Misc.Feature), "None"), exclude = NULL)) %>%
mutate(Mo.Sold = as.factor(Mo.Sold)) %>%
mutate(Yr.Sold = as.factor(Yr.Sold)) %>%
dplyr::select(-Garage.Yr.Blt) %>%
mutate(Condition.1 = as.character(Condition.1)) %>%
mutate(Kitchen.Qual=plyr::mapvalues(Kitchen.Qual, from = c("Po", "Fa", "TA","Gd", "Ex" ), to = c("1", "2", "3", "4", "5"))) %>%
mutate(Kitchen.Qual = as.numeric(as.character(Kitchen.Qual))) %>%
mutate(Heating.QC=plyr::mapvalues(Heating.QC, from = c("Po", "Fa", "TA","Gd", "Ex" ), to = c("1", "2", "3", "4", "5"))) %>%
mutate(Heating.QC = as.numeric(as.character(Heating.QC))) %>%
mutate(Bsmt.Qual = droplevels(Bsmt.Qual)) %>%
mutate(Functional = droplevels(Functional)) %>%
mutate(Roof.Matl = droplevels(Roof.Matl))
ind_rail<-which(data$Condition.1=="RRNn" | data$Condition.1=="RRAn" | data$Condition.1=="RRNe" | data$Condition.1=="RRAe")
ind_park<-which(data$Condition.1=="PosN" | data$Condition.1=="PosA")
data$Condition.1[ind_rail]<-"Rail"
data$Condition.1[ind_park]<-"Park"
data = data %>%
mutate(Condition.1 = factor(Condition.1)) %>%
mutate(Condition.1 = relevel(Condition.1, ref="Norm"))
data_test=data
data_test$Bsmt.Exposure[which(data_test$Bsmt.Exposure=="")]<-"None"
data_test$Bsmt.Exposure<-droplevels(data_test$Bsmt.Exposure)
data_test$Pool.Area<-data_test$Pool.Area+1
data_test$Total.Bsmt.SF<-data_test$Total.Bsmt.SF+1
# extract the truth
Y = data_test$price
# make prediction based on specific model
Yhat = predict(model.BIC, newdata=data_test, interval="predict")
# depending on the response transformation
Yhat = exp(Yhat)
# name dataframe as predictions! DO NOT CHANGE
predictions = as.data.frame(Yhat)
predictions$PID = data_test$PID
save(predictions, file="predict.Rdata")
performance(Y, Yhat)# Lasso for the simplified model (from BIC)
library(glmnet)
X.train = model.matrix(log(price) ~ log(Lot.Area) + Neighborhood + Condition.1 +
Overall.Qual + Overall.Cond + HouseAge + Foundation + Bsmt.Qual +
Bsmt.Exposure + log(Total.Bsmt.SF) + Heating.QC + Central.Air +
Baths + Bedroom.AbvGr + Kitchen.AbvGr + Kitchen.Qual + Functional +
Fireplaces + Garage.Cars + Paved.Drive + log(Pool.Area) +
log(TotalSq), data = data_train)
X.test = model.matrix(log(price) ~ log(Lot.Area) + Neighborhood + Condition.1 +
Overall.Qual + Overall.Cond + HouseAge + Foundation + Bsmt.Qual +
Bsmt.Exposure + log(Total.Bsmt.SF) + Heating.QC + Central.Air +
Baths + Bedroom.AbvGr + Kitchen.AbvGr + Kitchen.Qual + Functional +
Fireplaces + Garage.Cars + Paved.Drive + log(Pool.Area) +
log(TotalSq), data = data_test)
model.lasso = glmnet(X.train, log(data_train$price), alpha=1)
cv.lasso = cv.glmnet(X.train, log(data_train$price), alpha=1)
yhat.lasso = exp(predict(model.lasso, s=cv.lasso$lambda.min, type="response", newx = X.test))
sqrt(mean(((yhat.lasso)-data_test$price)^2))
# Ridge for the simplified model (from BIC)
model.ridge = glmnet(X.train, log(data_train$price), alpha=0)
cv.ridge = cv.glmnet(X.train, log(data_train$price), alpha=0)
yhat.ridge = predict(model.ridge, s=cv.ridge$lambda.min, type="response", newx = X.test)
sqrt(mean((exp(yhat.ridge)-data_test$price)^2))# BMA for simple model (from BIC)
library(BAS)
model.bma = bas.lm(log(price) ~ log(Lot.Area) + Neighborhood + Condition.1 +
Overall.Qual + Overall.Cond + HouseAge + Foundation + Bsmt.Qual +
Bsmt.Exposure + log(Total.Bsmt.SF) + Heating.QC + Central.Air +
Baths + Bedroom.AbvGr + Kitchen.AbvGr + Kitchen.Qual + Functional +
Fireplaces + Garage.Cars + Paved.Drive + log(Pool.Area) +
log(TotalSq), data=data_train, prior="g-prior", a=nrow(data_train), modelprior=uniform(), method="MCMC", MCMC.iterations = 200000, thin = 20)
y_pred<-predict.bas(model.bma,newdata=data_test, estimator="BPM", prediction=TRUE, se.fit=TRUE)
y_hat<-exp(confint.pred.bas(y_pred))
y_hat= y_hat[,c(3,1,2)]
performance(data_test$price, y_hat)
library(treemap)
tree.data = data_train %>% mutate(price.bin = ifelse(price <= 50000, "<= $50,000", ifelse(price > 50000 & price <= 100000, "$50,000-100,000", ifelse(price > 100000 & price <= 200000, "$100,000-200,000", ifelse(price > 200000 & price <= 300000, "$200,000-300,000", ifelse(price > 300000 & price <= 400000, "$300,000-400,000", ifelse(price > 400000 & price <= 500000, "$400,000-500,000", "> $500,000")))))))
area.plot = treemap(tree.data, index="price.bin", vSize="Lot.Area", type="index", palette="YlGn", algorithm="pivotSize", title="Effect of Lot Area on Price Range", fontsize.title = 14)
data_train$Overall.Qual = as.numeric(data_train$Overall.Qual)
quality.plot = ggplot(train, aes(x = Overall.Qual, y=price)) + geom_smooth(color="gold3") + geom_count(color="red3") + theme_minimal()
plot(quality.plot)
Model Evaluation on Test Data
Create predicted values for price using your model using the testing data
load("ames_test.Rdata")
#Variables with NA's and their proportion of missing data
miss = apply(is.na(ames_train), 2, sum)
miss_prop = round(miss[miss>0]/nrow(ames_train), 3)
print(miss_prop)
which(miss_prop>0.5) # four features have greater than 50% of data missing
#Created binary variables for whether or not a house as an alley, pool, fence, misc. feature, fireplace, basement or garage, as I thought it may be more meaningful- Tom
#May consider adding Lot.Frontage back in, as it only had 286 missing
#Have to decide between filtering out N/A's for different Garage variables and only including 'has.garage'
#Have to decide between filtering out N/A's for different Basement variables and only including 'has.basement'
test <- ames_test %>% filter(!is.na(Lot.Frontage)) %>%
mutate(Alley = factor(Alley, levels = levels(addNA(Alley)), labels = c(levels(Alley), "None"), exclude = NULL)) %>%
mutate(HouseAge = Yr.Sold- pmax(Year.Built, Year.Remod.Add)) %>%
filter(!is.na(Mas.Vnr.Area)) %>%
mutate(Bsmt.YN = !is.na(Bsmt.Qual)) %>%
mutate(Bsmt.Qual = factor(Bsmt.Qual, levels = levels(addNA(Bsmt.Qual)), labels = c(levels(Bsmt.Qual), "NoBa"), exclude = NULL)) %>%
mutate(Bsmt.Cond = factor(Bsmt.Cond, levels = levels(addNA(Bsmt.Cond)), labels = c(levels(Bsmt.Cond), "NoBa"), exclude = NULL)) %>%
mutate(Bsmt.Exposure = factor(Bsmt.Exposure, levels = levels(addNA(Bsmt.Exposure)), labels = c(levels(Bsmt.Exposure), "NoBa"), exclude = NULL)) %>%
mutate(BsmtFin.Type.1= factor(BsmtFin.Type.1, levels = levels(addNA(BsmtFin.Type.1)), labels = c(levels(BsmtFin.Type.1), "NoBa"), exclude = NULL)) %>%
mutate(BsmtFin.Type.2= factor(BsmtFin.Type.2, levels = levels(addNA(BsmtFin.Type.2)), labels = c(levels(BsmtFin.Type.2), "NoBa"), exclude = NULL)) %>%
mutate(X12.SF= X1st.Flr.SF+ X2nd.Flr.SF) %>%
filter(!is.na(Bsmt.Full.Bath)) %>%
filter(!is.na(Bsmt.Half.Bath)) %>%
mutate(Baths = Bsmt.Full.Bath + 0.5*Bsmt.Half.Bath + Full.Bath + 0.5*Half.Bath) %>%
mutate(Fireplace.YN = Fireplaces>0) %>%
mutate(Fireplace.Qu = factor(Fireplace.Qu, levels = levels(addNA(Fireplace.Qu)), labels = c(levels(Fireplace.Qu), "None"), exclude = NULL)) %>%
mutate(Garage.YN = !is.na(Garage.Cond)) %>%
mutate(Garage.Type = factor(Garage.Type, levels = levels(addNA(Garage.Type)), labels = c(levels(Garage.Type), "None"), exclude = NULL)) %>%
mutate(Garage.Finish = factor(Garage.Finish, levels = levels(addNA(Garage.Finish)), labels = c(levels(Garage.Finish), "None"), exclude = NULL)) %>%
mutate(Garage.Qual = factor(Garage.Qual, levels = levels(addNA(Garage.Qual)), labels = c(levels(Garage.Qual), "None"), exclude = NULL)) %>%
mutate(Garage.Cond = factor(Garage.Cond, levels = levels(addNA(Garage.Cond)), labels = c(levels(Garage.Cond), "None"), exclude = NULL)) %>%
mutate(Porch.Area = Wood.Deck.SF+ Open.Porch.SF+Enclosed.Porch+X3Ssn.Porch + Screen.Porch) %>%
mutate(Pool.YN = Pool.Area>0) %>%
mutate(Pool.QC = factor(Pool.QC, levels = levels(addNA(Pool.QC)), labels = c(levels(Pool.QC), "None"), exclude = NULL)) %>%
mutate(Fence = factor(Fence, levels = levels(addNA(Fence)), labels = c(levels(Fence), "None"), exclude = NULL)) %>%
mutate(Misc.Feature = factor(Misc.Feature, levels = levels(addNA(Misc.Feature)), labels = c(levels(Misc.Feature), "None"), exclude = NULL)) %>%
mutate(Mo.Sold = as.factor(Mo.Sold)) %>%
mutate(Yr.Sold = as.factor(Yr.Sold)) %>%
dplyr::select(-Garage.Yr.Blt) %>% filter(Condition.1 != "RRNe") %>% filter(Kitchen.Qual != "Po")
You should save your predictions in a dataframe with columns for PID (property identifier), fit, predicted values on the test data, and where possible lwr and upr, lower and upper 95% interval estimates for predicting price.
Y = test$price
Yhat = exp(Yhat)
#Bias
mean(Yhat[,1] - Y)
#Maximum Deviation
max(abs(Y - Yhat[,1]))
#Mean Absolute Deviation
mean(abs(Y - Yhat[,1]))
#RMSE
sqrt(mean((Y - Yhat[,1])^2))
#Coverage
mean(Yhat[,"lwr"] < Y & Yhat[,"upr"] > Y)
# name dataframe as predictions! DO NOT CHANGE
predictions = as.data.frame(Yhat)
predictions$PID = test$PID
save(predictions, file="predict.Rdata")Your models will be evaluated on the following criteria on the test data:
* Bias: Average (Yhat-Y) positive values indicate the model tends to overestimate price (on average) while negative values indicate the model tends to underestimate price.
* Maximum Deviation: Max |Y-Yhat| - identifies the worst prediction made in the validation data set.
* Mean Absolute Deviation: Average |Y-Yhat| - the average error (regardless of sign).
Root Mean Square Error: Sqrt Average (Y-Yhat)^2
Coverage: Average( lwr < Y < upr)
In order to have a passing wercker badge, your file for predictions needs to be the same length as the test data, with three columns: fitted values, lower CI and upper CI values in that order with names, fit, lwr, and upr respectively.
You will be able to see your scores on the score board (coming soon!). They will be initialized by a predction based on the mean in the training data.
Model Check - Test your prediction on the first observation in the training and test data set to make sure that the model gives a reasonable answer and include this in a supplement of your report. This should be done BY HAND using a calculator (this means use the raw data from the original dataset and manually calculate all transformations and interactions with your calculator)! Models that do not give reasonable answers will be given a minimum 2 letter grade reduction. Also be careful as you cannot use certain transformations [log or inverse x] if a variable has values of 0.
Part II: Complex Model
In this part you may go all out for constructing a best fitting model for predicting housing prices using methods that we have covered this semester. You should feel free to to create any new variables (such as quadratic, interaction, or indicator variables, splines, etc). The variable TotalSq = X1st.Flr.SF+X2nd.Flr.SF was added to the dataframe (that does not include basement area, so you may improve on this. A relative grade is assigned by comparing your fit on the test set to that of your fellow students with bonus points awarded to those who substantially exceed their fellow students and point reductions occurring for models which fit exceedingly poorly.
library(glmnet)
#Design matrices
X.train = model.matrix(log(price) ~ log(Lot.Area) + Neighborhood + Condition.1 +
Overall.Qual + Overall.Cond + HouseAge + Foundation + Total.Bsmt.SF +
Central.Air + log(X12.SF) + Baths + Kitchen.AbvGr + Kitchen.Qual +
Functional + Fireplaces + Garage.Cars + Paved.Drive + Bsmt.YN:Bsmt.Exposure,
data = data)
X.test = model.matrix(log(price) ~ log(Lot.Area) + Neighborhood + Condition.1 +
Overall.Qual + Overall.Cond + HouseAge + Foundation + Total.Bsmt.SF +
Central.Air + log(X12.SF) + Baths + Kitchen.AbvGr + Kitchen.Qual +
Functional + Fireplaces + Garage.Cars + Paved.Drive + Bsmt.YN:Bsmt.Exposure,
data = test)
#Fit lasso regression
price.lasso = glmnet(X.train, log(data$price), alpha=1)
cv.lasso = cv.glmnet(X.train, log(data$price), alpha=1)
#Obtain prediction on test data
yhat.lasso.test = predict(app.lasso, s=cv.lasso$lambda.min, type="response", newx = X.test)
#Compute RMSE for test data
rmse.lasso.test = rmse(test$price, exp(yhat.lasso.test))
rmse.tab$lasso<-rmse.lasso.testUpdate your predictions using your complex model to provide point estimates and CI.
You may iterate here as much as you like exploring different models until you are satisfied with your results.
Bayesian lasso
coef(housing.lasso)
1170 x 1 sparse Matrix of class "dgCMatrix"
s0
(Intercept) -2.481733e+05
(Intercept) .
log(Lot.Area) -2.820674e+03
NeighborhoodBlueste .
NeighborhoodBrDale -2.071429e+04
NeighborhoodBrkSide -1.922483e+04
NeighborhoodClearCr 4.098550e+04
NeighborhoodCollgCr -2.140941e+03
NeighborhoodCrawfor -2.194616e+04
NeighborhoodEdwards -3.741877e+04
NeighborhoodGilbert -2.063168e+04
NeighborhoodGreens -1.468658e+04
NeighborhoodGrnHill 7.989575e+04
NeighborhoodIDOTRR -3.370340e+04
NeighborhoodLandmrk .
NeighborhoodMeadowV -7.435215e+04
NeighborhoodMitchel 7.567808e+02
NeighborhoodNAmes 2.532327e+04
NeighborhoodNoRidge 5.960950e+04
NeighborhoodNPkVill .
NeighborhoodNridgHt 6.494221e+04
NeighborhoodNWAmes .
NeighborhoodOldTown -1.200522e+04
NeighborhoodSawyer -2.410915e+04
NeighborhoodSawyerW -3.278347e+04
NeighborhoodSomerst 1.555067e+04
NeighborhoodStoneBr 3.757409e+04
NeighborhoodSWISU 7.946636e+03
NeighborhoodTimber 4.623122e+03
NeighborhoodVeenker .
Condition.1Artery -5.160022e+03
Condition.1Feedr 7.435678e+03
Condition.1Park 6.811294e+03
Condition.1Rail -2.466063e+04
Overall.Qual -8.687108e+03
Overall.Cond -1.933976e+04
HouseAge -5.417731e+02
FoundationCBlock 3.181746e+02
FoundationPConc 6.481395e+03
FoundationSlab -2.794967e+04
FoundationStone -9.737201e+00
FoundationWood .
Total.Bsmt.SF 1.086050e+01
Central.AirY -4.153187e+04
log(X12.SF) 4.482594e+04
Baths -3.003026e+03
Kitchen.AbvGr 9.251757e+02
Kitchen.Qual 6.304757e+02
FunctionalMaj2 -3.203463e+04
FunctionalMin1 7.379779e+04
FunctionalMin2 7.673434e+04
FunctionalMod -6.627107e+03
FunctionalTyp 2.093706e+04
Fireplaces .
Garage.Cars -9.906074e+03
Paved.DriveP -8.263301e+03
Paved.DriveY 1.011677e+04
log(Lot.Area):NeighborhoodBlueste 1.969465e+03
log(Lot.Area):NeighborhoodBrDale -1.439665e+03
log(Lot.Area):NeighborhoodBrkSide 5.167852e+03
log(Lot.Area):NeighborhoodClearCr 1.487235e+00
log(Lot.Area):NeighborhoodCollgCr -1.866509e+03
log(Lot.Area):NeighborhoodCrawfor 1.481544e+02
log(Lot.Area):NeighborhoodEdwards 2.345952e+03
log(Lot.Area):NeighborhoodGilbert -4.651634e+03
log(Lot.Area):NeighborhoodGreens .
log(Lot.Area):NeighborhoodGrnHill 8.574359e+02
log(Lot.Area):NeighborhoodIDOTRR 1.017452e+03
log(Lot.Area):NeighborhoodLandmrk -1.469301e+03
log(Lot.Area):NeighborhoodMeadowV -4.785444e+03
log(Lot.Area):NeighborhoodMitchel -2.561962e+03
log(Lot.Area):NeighborhoodNAmes -1.370296e+03
log(Lot.Area):NeighborhoodNoRidge 3.140092e+04
log(Lot.Area):NeighborhoodNPkVill 1.722498e+03
log(Lot.Area):NeighborhoodNridgHt -4.336223e+03
log(Lot.Area):NeighborhoodNWAmes 1.006834e+01
log(Lot.Area):NeighborhoodOldTown -4.135321e+03
log(Lot.Area):NeighborhoodSawyer -6.372042e+03
log(Lot.Area):NeighborhoodSawyerW -4.773142e+02
log(Lot.Area):NeighborhoodSomerst 5.610437e+03
log(Lot.Area):NeighborhoodStoneBr 5.260326e+03
log(Lot.Area):NeighborhoodSWISU 2.377089e+02
log(Lot.Area):NeighborhoodTimber 7.785457e+03
log(Lot.Area):NeighborhoodVeenker 5.449607e+03
log(Lot.Area):Condition.1Artery 1.906080e+03
log(Lot.Area):Condition.1Feedr -9.609245e+02
log(Lot.Area):Condition.1Park 2.508378e+03
log(Lot.Area):Condition.1Rail .
log(Lot.Area):Overall.Qual 9.696353e+02
log(Lot.Area):Overall.Cond 3.072593e+02
log(Lot.Area):HouseAge 2.060536e+01
log(Lot.Area):FoundationCBlock 1.799227e+03
log(Lot.Area):FoundationPConc 1.972279e+02
log(Lot.Area):FoundationSlab -4.428848e+03
log(Lot.Area):FoundationStone 4.257674e+02
log(Lot.Area):FoundationWood .
log(Lot.Area):Total.Bsmt.SF 3.608480e+00
log(Lot.Area):Central.AirY -6.891702e+03
log(Lot.Area):log(X12.SF) 2.298431e+03
log(Lot.Area):Baths 7.714103e+02
log(Lot.Area):Kitchen.AbvGr -2.928009e+03
log(Lot.Area):Kitchen.Qual 3.625858e+02
log(Lot.Area):FunctionalMaj2 .
log(Lot.Area):FunctionalMin1 2.939519e+03
log(Lot.Area):FunctionalMin2 3.815929e+03
log(Lot.Area):FunctionalMod -9.929820e-01
log(Lot.Area):FunctionalTyp -5.721209e+02
log(Lot.Area):Fireplaces .
log(Lot.Area):Garage.Cars -4.124218e+02
log(Lot.Area):Paved.DriveP -5.198722e+03
log(Lot.Area):Paved.DriveY -3.443184e+03
NeighborhoodBlueste:Condition.1Artery .
NeighborhoodBrDale:Condition.1Artery .
NeighborhoodBrkSide:Condition.1Artery .
NeighborhoodClearCr:Condition.1Artery .
NeighborhoodCollgCr:Condition.1Artery .
NeighborhoodCrawfor:Condition.1Artery .
NeighborhoodEdwards:Condition.1Artery 1.865301e+04
NeighborhoodGilbert:Condition.1Artery .
NeighborhoodGreens:Condition.1Artery .
NeighborhoodGrnHill:Condition.1Artery .
NeighborhoodIDOTRR:Condition.1Artery -2.866272e+04
NeighborhoodLandmrk:Condition.1Artery .
NeighborhoodMeadowV:Condition.1Artery .
NeighborhoodMitchel:Condition.1Artery .
NeighborhoodNAmes:Condition.1Artery -7.176780e+03
NeighborhoodNoRidge:Condition.1Artery .
NeighborhoodNPkVill:Condition.1Artery .
NeighborhoodNridgHt:Condition.1Artery .
NeighborhoodNWAmes:Condition.1Artery .
NeighborhoodOldTown:Condition.1Artery .
NeighborhoodSawyer:Condition.1Artery .
NeighborhoodSawyerW:Condition.1Artery .
NeighborhoodSomerst:Condition.1Artery .
NeighborhoodStoneBr:Condition.1Artery .
NeighborhoodSWISU:Condition.1Artery -2.692082e+04
NeighborhoodTimber:Condition.1Artery .
NeighborhoodVeenker:Condition.1Artery .
NeighborhoodBlueste:Condition.1Feedr .
NeighborhoodBrDale:Condition.1Feedr .
NeighborhoodBrkSide:Condition.1Feedr 1.384133e+04
NeighborhoodClearCr:Condition.1Feedr -2.708729e+04
NeighborhoodCollgCr:Condition.1Feedr .
NeighborhoodCrawfor:Condition.1Feedr 1.660394e+03
NeighborhoodEdwards:Condition.1Feedr 1.066244e+04
NeighborhoodGilbert:Condition.1Feedr -1.520833e+04
NeighborhoodGreens:Condition.1Feedr .
NeighborhoodGrnHill:Condition.1Feedr .
NeighborhoodIDOTRR:Condition.1Feedr -2.489069e+04
NeighborhoodLandmrk:Condition.1Feedr .
NeighborhoodMeadowV:Condition.1Feedr .
NeighborhoodMitchel:Condition.1Feedr .
NeighborhoodNAmes:Condition.1Feedr -2.201557e+00
NeighborhoodNoRidge:Condition.1Feedr .
NeighborhoodNPkVill:Condition.1Feedr .
NeighborhoodNridgHt:Condition.1Feedr .
NeighborhoodNWAmes:Condition.1Feedr 9.716783e+03
NeighborhoodOldTown:Condition.1Feedr 1.707263e+04
NeighborhoodSawyer:Condition.1Feedr 4.306266e+03
NeighborhoodSawyerW:Condition.1Feedr 2.616033e+04
NeighborhoodSomerst:Condition.1Feedr 1.167382e+04
NeighborhoodStoneBr:Condition.1Feedr .
NeighborhoodSWISU:Condition.1Feedr 2.686645e+04
NeighborhoodTimber:Condition.1Feedr .
NeighborhoodVeenker:Condition.1Feedr 3.782459e+04
NeighborhoodBlueste:Condition.1Park .
NeighborhoodBrDale:Condition.1Park .
NeighborhoodBrkSide:Condition.1Park .
NeighborhoodClearCr:Condition.1Park .
NeighborhoodCollgCr:Condition.1Park -5.223636e+03
NeighborhoodCrawfor:Condition.1Park .
NeighborhoodEdwards:Condition.1Park .
NeighborhoodGilbert:Condition.1Park .
NeighborhoodGreens:Condition.1Park .
NeighborhoodGrnHill:Condition.1Park .
NeighborhoodIDOTRR:Condition.1Park .
NeighborhoodLandmrk:Condition.1Park .
NeighborhoodMeadowV:Condition.1Park .
NeighborhoodMitchel:Condition.1Park .
NeighborhoodNAmes:Condition.1Park -8.382130e+02
NeighborhoodNoRidge:Condition.1Park .
NeighborhoodNPkVill:Condition.1Park .
NeighborhoodNridgHt:Condition.1Park -8.323452e+02
NeighborhoodNWAmes:Condition.1Park .
NeighborhoodOldTown:Condition.1Park .
NeighborhoodSawyer:Condition.1Park .
NeighborhoodSawyerW:Condition.1Park .
NeighborhoodSomerst:Condition.1Park -9.615583e+03
NeighborhoodStoneBr:Condition.1Park -2.173948e+04
NeighborhoodSWISU:Condition.1Park .
NeighborhoodTimber:Condition.1Park .
NeighborhoodVeenker:Condition.1Park .
NeighborhoodBlueste:Condition.1Rail .
NeighborhoodBrDale:Condition.1Rail .
NeighborhoodBrkSide:Condition.1Rail 1.579995e+04
NeighborhoodClearCr:Condition.1Rail .
NeighborhoodCollgCr:Condition.1Rail .
NeighborhoodCrawfor:Condition.1Rail .
NeighborhoodEdwards:Condition.1Rail .
NeighborhoodGilbert:Condition.1Rail -3.785934e+04
NeighborhoodGreens:Condition.1Rail .
NeighborhoodGrnHill:Condition.1Rail .
NeighborhoodIDOTRR:Condition.1Rail 2.212995e+04
NeighborhoodLandmrk:Condition.1Rail .
NeighborhoodMeadowV:Condition.1Rail .
NeighborhoodMitchel:Condition.1Rail .
NeighborhoodNAmes:Condition.1Rail .
NeighborhoodNoRidge:Condition.1Rail .
NeighborhoodNPkVill:Condition.1Rail .
NeighborhoodNridgHt:Condition.1Rail .
NeighborhoodNWAmes:Condition.1Rail 1.051407e+04
NeighborhoodOldTown:Condition.1Rail 4.317082e+04
NeighborhoodSawyer:Condition.1Rail 6.821910e+03
NeighborhoodSawyerW:Condition.1Rail -2.095610e+04
NeighborhoodSomerst:Condition.1Rail 2.077964e+03
NeighborhoodStoneBr:Condition.1Rail .
NeighborhoodSWISU:Condition.1Rail .
NeighborhoodTimber:Condition.1Rail .
NeighborhoodVeenker:Condition.1Rail .
NeighborhoodBlueste:Overall.Qual 1.278702e+04
NeighborhoodBrDale:Overall.Qual 1.280317e+03
NeighborhoodBrkSide:Overall.Qual 4.509090e+02
NeighborhoodClearCr:Overall.Qual 9.120932e+03
NeighborhoodCollgCr:Overall.Qual -2.887457e+03
NeighborhoodCrawfor:Overall.Qual 2.288150e+03
NeighborhoodEdwards:Overall.Qual -7.863646e+03
NeighborhoodGilbert:Overall.Qual 3.356545e+03
NeighborhoodGreens:Overall.Qual 1.387156e+03
NeighborhoodGrnHill:Overall.Qual 1.129114e+03
NeighborhoodIDOTRR:Overall.Qual -2.210951e+03
NeighborhoodLandmrk:Overall.Qual .
NeighborhoodMeadowV:Overall.Qual -6.782732e+03
NeighborhoodMitchel:Overall.Qual -6.464510e+03
NeighborhoodNAmes:Overall.Qual -2.966498e+03
NeighborhoodNoRidge:Overall.Qual -2.383321e+03
NeighborhoodNPkVill:Overall.Qual -1.126953e+04
NeighborhoodNridgHt:Overall.Qual 6.547708e+02
NeighborhoodNWAmes:Overall.Qual -5.969528e+03
NeighborhoodOldTown:Overall.Qual -1.789256e+03
NeighborhoodSawyer:Overall.Qual -4.263807e+03
NeighborhoodSawyerW:Overall.Qual -2.596599e+03
NeighborhoodSomerst:Overall.Qual -8.919578e+02
NeighborhoodStoneBr:Overall.Qual -1.173775e+04
NeighborhoodSWISU:Overall.Qual -5.900090e+02
NeighborhoodTimber:Overall.Qual .
NeighborhoodVeenker:Overall.Qual -1.926608e+03
NeighborhoodBlueste:Overall.Cond 5.800491e+03
NeighborhoodBrDale:Overall.Cond 4.294752e+03
NeighborhoodBrkSide:Overall.Cond -6.818636e+02
NeighborhoodClearCr:Overall.Cond 3.576969e+03
NeighborhoodCollgCr:Overall.Cond 6.087199e+03
NeighborhoodCrawfor:Overall.Cond -4.536931e+03
NeighborhoodEdwards:Overall.Cond 8.003291e+02
NeighborhoodGilbert:Overall.Cond 6.970022e+03
NeighborhoodGreens:Overall.Cond 8.502759e+03
NeighborhoodGrnHill:Overall.Cond .
NeighborhoodIDOTRR:Overall.Cond 3.014450e+03
NeighborhoodLandmrk:Overall.Cond .
NeighborhoodMeadowV:Overall.Cond -1.986557e+03
NeighborhoodMitchel:Overall.Cond -1.014271e+03
NeighborhoodNAmes:Overall.Cond -2.895080e+02
NeighborhoodNoRidge:Overall.Cond -1.245017e+04
NeighborhoodNPkVill:Overall.Cond 5.098328e+03
NeighborhoodNridgHt:Overall.Cond -1.884113e+03
NeighborhoodNWAmes:Overall.Cond 3.578541e+03
NeighborhoodOldTown:Overall.Cond 7.459405e+02
NeighborhoodSawyer:Overall.Cond 3.681778e+03
NeighborhoodSawyerW:Overall.Cond 5.527394e+03
NeighborhoodSomerst:Overall.Cond -6.758819e+03
NeighborhoodStoneBr:Overall.Cond 2.448421e+02
NeighborhoodSWISU:Overall.Cond -6.103562e+03
NeighborhoodTimber:Overall.Cond -7.422249e+03
NeighborhoodVeenker:Overall.Cond -1.897816e+04
NeighborhoodBlueste:HouseAge -1.585781e+02
NeighborhoodBrDale:HouseAge 2.817410e+02
NeighborhoodBrkSide:HouseAge 2.616997e+02
NeighborhoodClearCr:HouseAge 6.843583e+02
NeighborhoodCollgCr:HouseAge 4.756420e+02
NeighborhoodCrawfor:HouseAge 1.009226e+00
NeighborhoodEdwards:HouseAge -3.853751e+01
NeighborhoodGilbert:HouseAge -8.908637e+01
NeighborhoodGreens:HouseAge 9.156589e+02
NeighborhoodGrnHill:HouseAge 1.861608e+02
NeighborhoodIDOTRR:HouseAge 3.298973e+02
NeighborhoodLandmrk:HouseAge .
NeighborhoodMeadowV:HouseAge 1.151213e+02
NeighborhoodMitchel:HouseAge 5.838366e+01
NeighborhoodNAmes:HouseAge 1.942339e+02
NeighborhoodNoRidge:HouseAge -6.236985e+02
NeighborhoodNPkVill:HouseAge 3.952497e+02
NeighborhoodNridgHt:HouseAge -1.192641e+03
NeighborhoodNWAmes:HouseAge -5.735172e+00
NeighborhoodOldTown:HouseAge 2.883042e+02
NeighborhoodSawyer:HouseAge 3.830832e+02
NeighborhoodSawyerW:HouseAge 2.092397e+02
NeighborhoodSomerst:HouseAge 1.433830e+03
NeighborhoodStoneBr:HouseAge -1.529292e+03
NeighborhoodSWISU:HouseAge -4.789039e+01
NeighborhoodTimber:HouseAge -2.824966e+02
NeighborhoodVeenker:HouseAge -1.591223e+03
NeighborhoodBlueste:FoundationCBlock -7.670598e+04
NeighborhoodBrDale:FoundationCBlock 1.194988e+03
NeighborhoodBrkSide:FoundationCBlock 1.312685e+04
NeighborhoodClearCr:FoundationCBlock -1.095257e+05
NeighborhoodCollgCr:FoundationCBlock -1.305436e+04
NeighborhoodCrawfor:FoundationCBlock 3.376332e+04
NeighborhoodEdwards:FoundationCBlock 2.799478e+04
NeighborhoodGilbert:FoundationCBlock -4.657422e+04
NeighborhoodGreens:FoundationCBlock -3.348628e+04
NeighborhoodGrnHill:FoundationCBlock 3.321382e+03
NeighborhoodIDOTRR:FoundationCBlock 1.742947e+04
NeighborhoodLandmrk:FoundationCBlock .
NeighborhoodMeadowV:FoundationCBlock 9.142792e+04
NeighborhoodMitchel:FoundationCBlock 6.711466e+04
NeighborhoodNAmes:FoundationCBlock 1.799054e+04
NeighborhoodNoRidge:FoundationCBlock .
NeighborhoodNPkVill:FoundationCBlock 1.880958e+04
NeighborhoodNridgHt:FoundationCBlock .
NeighborhoodNWAmes:FoundationCBlock 3.850965e+03
NeighborhoodOldTown:FoundationCBlock 1.375454e+04
NeighborhoodSawyer:FoundationCBlock 1.177385e+04
NeighborhoodSawyerW:FoundationCBlock 1.285960e+03
NeighborhoodSomerst:FoundationCBlock .
NeighborhoodStoneBr:FoundationCBlock 3.484069e+04
NeighborhoodSWISU:FoundationCBlock 5.202907e+03
NeighborhoodTimber:FoundationCBlock -5.418507e+04
NeighborhoodVeenker:FoundationCBlock 4.324409e+04
NeighborhoodBlueste:FoundationPConc .
NeighborhoodBrDale:FoundationPConc .
NeighborhoodBrkSide:FoundationPConc 1.555431e+04
NeighborhoodClearCr:FoundationPConc -7.458508e+04
NeighborhoodCollgCr:FoundationPConc .
NeighborhoodCrawfor:FoundationPConc 3.840414e+04
NeighborhoodEdwards:FoundationPConc 3.017124e+04
NeighborhoodGilbert:FoundationPConc 2.678701e+03
NeighborhoodGreens:FoundationPConc .
NeighborhoodGrnHill:FoundationPConc .
NeighborhoodIDOTRR:FoundationPConc 2.683037e+04
NeighborhoodLandmrk:FoundationPConc .
NeighborhoodMeadowV:FoundationPConc 9.974146e+04
NeighborhoodMitchel:FoundationPConc 6.000149e+04
NeighborhoodNAmes:FoundationPConc 2.005812e+04
NeighborhoodNoRidge:FoundationPConc -2.399086e+05
NeighborhoodNPkVill:FoundationPConc .
NeighborhoodNridgHt:FoundationPConc .
NeighborhoodNWAmes:FoundationPConc 1.183912e+04
NeighborhoodOldTown:FoundationPConc 1.507170e+04
NeighborhoodSawyer:FoundationPConc 1.339285e+04
NeighborhoodSawyerW:FoundationPConc -1.419901e+03
NeighborhoodSomerst:FoundationPConc -3.273427e+03
NeighborhoodStoneBr:FoundationPConc 6.186384e+03
NeighborhoodSWISU:FoundationPConc 2.033934e+04
NeighborhoodTimber:FoundationPConc -3.695573e+04
NeighborhoodVeenker:FoundationPConc 3.070993e+04
NeighborhoodBlueste:FoundationSlab .
NeighborhoodBrDale:FoundationSlab .
NeighborhoodBrkSide:FoundationSlab 2.761503e+04
NeighborhoodClearCr:FoundationSlab .
NeighborhoodCollgCr:FoundationSlab .
NeighborhoodCrawfor:FoundationSlab .
NeighborhoodEdwards:FoundationSlab 5.309315e+04
NeighborhoodGilbert:FoundationSlab .
NeighborhoodGreens:FoundationSlab .
NeighborhoodGrnHill:FoundationSlab .
NeighborhoodIDOTRR:FoundationSlab 3.810880e+04
NeighborhoodLandmrk:FoundationSlab .
NeighborhoodMeadowV:FoundationSlab .
NeighborhoodMitchel:FoundationSlab 5.955324e+04
NeighborhoodNAmes:FoundationSlab 1.810422e+04
NeighborhoodNoRidge:FoundationSlab .
NeighborhoodNPkVill:FoundationSlab .
NeighborhoodNridgHt:FoundationSlab .
NeighborhoodNWAmes:FoundationSlab .
NeighborhoodOldTown:FoundationSlab 1.051785e+04
NeighborhoodSawyer:FoundationSlab -1.883762e+04
NeighborhoodSawyerW:FoundationSlab 6.456851e+03
NeighborhoodSomerst:FoundationSlab .
NeighborhoodStoneBr:FoundationSlab .
NeighborhoodSWISU:FoundationSlab 1.007064e+05
NeighborhoodTimber:FoundationSlab .
NeighborhoodVeenker:FoundationSlab .
NeighborhoodBlueste:FoundationStone .
NeighborhoodBrDale:FoundationStone .
NeighborhoodBrkSide:FoundationStone .
NeighborhoodClearCr:FoundationStone .
NeighborhoodCollgCr:FoundationStone .
NeighborhoodCrawfor:FoundationStone 1.418961e+04
NeighborhoodEdwards:FoundationStone .
NeighborhoodGilbert:FoundationStone .
NeighborhoodGreens:FoundationStone .
NeighborhoodGrnHill:FoundationStone .
NeighborhoodIDOTRR:FoundationStone -8.295564e+04
NeighborhoodLandmrk:FoundationStone .
NeighborhoodMeadowV:FoundationStone .
NeighborhoodMitchel:FoundationStone .
NeighborhoodNAmes:FoundationStone 3.948740e+03
NeighborhoodNoRidge:FoundationStone .
NeighborhoodNPkVill:FoundationStone .
NeighborhoodNridgHt:FoundationStone .
NeighborhoodNWAmes:FoundationStone .
NeighborhoodOldTown:FoundationStone .
NeighborhoodSawyer:FoundationStone .
NeighborhoodSawyerW:FoundationStone .
NeighborhoodSomerst:FoundationStone .
NeighborhoodStoneBr:FoundationStone .
NeighborhoodSWISU:FoundationStone .
NeighborhoodTimber:FoundationStone .
NeighborhoodVeenker:FoundationStone .
NeighborhoodBlueste:FoundationWood .
NeighborhoodBrDale:FoundationWood .
NeighborhoodBrkSide:FoundationWood .
NeighborhoodClearCr:FoundationWood .
NeighborhoodCollgCr:FoundationWood .
NeighborhoodCrawfor:FoundationWood .
NeighborhoodEdwards:FoundationWood .
NeighborhoodGilbert:FoundationWood -8.825003e+03
NeighborhoodGreens:FoundationWood .
NeighborhoodGrnHill:FoundationWood .
NeighborhoodIDOTRR:FoundationWood .
NeighborhoodLandmrk:FoundationWood .
NeighborhoodMeadowV:FoundationWood .
NeighborhoodMitchel:FoundationWood .
NeighborhoodNAmes:FoundationWood .
NeighborhoodNoRidge:FoundationWood .
NeighborhoodNPkVill:FoundationWood .
NeighborhoodNridgHt:FoundationWood .
NeighborhoodNWAmes:FoundationWood .
NeighborhoodOldTown:FoundationWood .
NeighborhoodSawyer:FoundationWood .
NeighborhoodSawyerW:FoundationWood .
NeighborhoodSomerst:FoundationWood .
NeighborhoodStoneBr:FoundationWood .
NeighborhoodSWISU:FoundationWood .
NeighborhoodTimber:FoundationWood .
NeighborhoodVeenker:FoundationWood .
NeighborhoodBlueste:Total.Bsmt.SF -6.195864e+01
NeighborhoodBrDale:Total.Bsmt.SF -3.025655e+01
NeighborhoodBrkSide:Total.Bsmt.SF 9.448265e-01
NeighborhoodClearCr:Total.Bsmt.SF 1.458647e+01
NeighborhoodCollgCr:Total.Bsmt.SF -7.984066e+00
NeighborhoodCrawfor:Total.Bsmt.SF -4.388974e+01
NeighborhoodEdwards:Total.Bsmt.SF -2.795649e-02
NeighborhoodGilbert:Total.Bsmt.SF .
NeighborhoodGreens:Total.Bsmt.SF -3.501130e+01
NeighborhoodGrnHill:Total.Bsmt.SF .
NeighborhoodIDOTRR:Total.Bsmt.SF -1.841197e+01
NeighborhoodLandmrk:Total.Bsmt.SF .
NeighborhoodMeadowV:Total.Bsmt.SF -1.120302e+00
NeighborhoodMitchel:Total.Bsmt.SF 1.876016e+01
NeighborhoodNAmes:Total.Bsmt.SF -1.316175e+01
NeighborhoodNoRidge:Total.Bsmt.SF -7.402320e+00
NeighborhoodNPkVill:Total.Bsmt.SF 1.540046e+01
NeighborhoodNridgHt:Total.Bsmt.SF 8.899627e+00
NeighborhoodNWAmes:Total.Bsmt.SF -1.560533e+01
NeighborhoodOldTown:Total.Bsmt.SF -1.072386e+01
NeighborhoodSawyer:Total.Bsmt.SF -5.284939e+00
NeighborhoodSawyerW:Total.Bsmt.SF -9.205049e+00
NeighborhoodSomerst:Total.Bsmt.SF 1.697050e+00
NeighborhoodStoneBr:Total.Bsmt.SF 6.252423e+01
NeighborhoodSWISU:Total.Bsmt.SF -5.135103e+01
NeighborhoodTimber:Total.Bsmt.SF 1.482535e-02
NeighborhoodVeenker:Total.Bsmt.SF 1.402837e+01
NeighborhoodBlueste:Central.AirY .
NeighborhoodBrDale:Central.AirY .
NeighborhoodBrkSide:Central.AirY 1.326769e+04
NeighborhoodClearCr:Central.AirY -2.613468e+04
NeighborhoodCollgCr:Central.AirY 2.291435e+03
NeighborhoodCrawfor:Central.AirY 3.228596e+04
NeighborhoodEdwards:Central.AirY 2.076208e+04
NeighborhoodGilbert:Central.AirY -7.812483e+03
NeighborhoodGreens:Central.AirY .
NeighborhoodGrnHill:Central.AirY 3.542000e+02
NeighborhoodIDOTRR:Central.AirY 3.945154e+04
NeighborhoodLandmrk:Central.AirY .
NeighborhoodMeadowV:Central.AirY 8.393022e+03
NeighborhoodMitchel:Central.AirY -2.922281e+03
NeighborhoodNAmes:Central.AirY -1.189453e+04
NeighborhoodNoRidge:Central.AirY .
NeighborhoodNPkVill:Central.AirY .
NeighborhoodNridgHt:Central.AirY -1.065700e+04
NeighborhoodNWAmes:Central.AirY 3.715909e+03
NeighborhoodOldTown:Central.AirY 2.967330e+04
NeighborhoodSawyer:Central.AirY 2.615836e+04
NeighborhoodSawyerW:Central.AirY -1.775825e+03
NeighborhoodSomerst:Central.AirY -3.837463e+03
NeighborhoodStoneBr:Central.AirY -6.277786e+04
NeighborhoodSWISU:Central.AirY 2.944683e+04
NeighborhoodTimber:Central.AirY -6.722770e+03
NeighborhoodVeenker:Central.AirY 1.648501e+03
NeighborhoodBlueste:log(X12.SF) .
NeighborhoodBrDale:log(X12.SF) 3.896258e+02
NeighborhoodBrkSide:log(X12.SF) -4.289714e+03
NeighborhoodClearCr:log(X12.SF) -8.923654e+03
NeighborhoodCollgCr:log(X12.SF) 6.495664e+02
NeighborhoodCrawfor:log(X12.SF) 1.438919e+04
NeighborhoodEdwards:log(X12.SF) -1.357257e+04
NeighborhoodGilbert:log(X12.SF) -1.225397e+02
NeighborhoodGreens:log(X12.SF) -3.341844e+02
NeighborhoodGrnHill:log(X12.SF) .
NeighborhoodIDOTRR:log(X12.SF) -1.735208e+04
..............................
........suppressing rows in show(); maybe adjust 'options(max.print= *, width = *)'
..............................
s0
NeighborhoodOldTown:FunctionalMin2 6.052447e+04
NeighborhoodSawyer:FunctionalMin2 4.161795e+04
NeighborhoodSawyerW:FunctionalMin2 -7.150591e+03
NeighborhoodSomerst:FunctionalMin2 -3.311685e+04
NeighborhoodStoneBr:FunctionalMin2 .
NeighborhoodSWISU:FunctionalMin2 2.798351e+04
NeighborhoodTimber:FunctionalMin2 .
NeighborhoodVeenker:FunctionalMin2 .
NeighborhoodBlueste:FunctionalMod .
NeighborhoodBrDale:FunctionalMod .
NeighborhoodBrkSide:FunctionalMod -4.555339e+04
NeighborhoodClearCr:FunctionalMod -5.515845e+04
NeighborhoodCollgCr:FunctionalMod .
NeighborhoodCrawfor:FunctionalMod .
NeighborhoodEdwards:FunctionalMod 2.113144e+04
NeighborhoodGilbert:FunctionalMod .
NeighborhoodGreens:FunctionalMod .
NeighborhoodGrnHill:FunctionalMod .
NeighborhoodIDOTRR:FunctionalMod 1.539571e+04
NeighborhoodLandmrk:FunctionalMod .
NeighborhoodMeadowV:FunctionalMod .
NeighborhoodMitchel:FunctionalMod .
NeighborhoodNAmes:FunctionalMod 2.030644e+02
NeighborhoodNoRidge:FunctionalMod .
NeighborhoodNPkVill:FunctionalMod .
NeighborhoodNridgHt:FunctionalMod .
NeighborhoodNWAmes:FunctionalMod .
NeighborhoodOldTown:FunctionalMod 3.754003e+04
NeighborhoodSawyer:FunctionalMod -1.365215e+04
NeighborhoodSawyerW:FunctionalMod -3.950334e+04
NeighborhoodSomerst:FunctionalMod .
NeighborhoodStoneBr:FunctionalMod 2.230404e+04
NeighborhoodSWISU:FunctionalMod .
NeighborhoodTimber:FunctionalMod .
NeighborhoodVeenker:FunctionalMod -8.205382e+02
NeighborhoodBlueste:FunctionalTyp 8.910605e+02
NeighborhoodBrDale:FunctionalTyp .
NeighborhoodBrkSide:FunctionalTyp -5.907216e+04
NeighborhoodClearCr:FunctionalTyp 1.007998e+04
NeighborhoodCollgCr:FunctionalTyp -2.905831e+03
NeighborhoodCrawfor:FunctionalTyp 9.723137e+03
NeighborhoodEdwards:FunctionalTyp 1.811535e+04
NeighborhoodGilbert:FunctionalTyp -1.038670e+02
NeighborhoodGreens:FunctionalTyp .
NeighborhoodGrnHill:FunctionalTyp .
NeighborhoodIDOTRR:FunctionalTyp 3.608659e+04
NeighborhoodLandmrk:FunctionalTyp .
NeighborhoodMeadowV:FunctionalTyp -2.435448e+04
NeighborhoodMitchel:FunctionalTyp 6.309683e+03
NeighborhoodNAmes:FunctionalTyp -1.030705e+04
NeighborhoodNoRidge:FunctionalTyp -6.959313e+04
NeighborhoodNPkVill:FunctionalTyp -7.885402e+02
NeighborhoodNridgHt:FunctionalTyp -3.785875e+04
NeighborhoodNWAmes:FunctionalTyp 4.227402e+04
NeighborhoodOldTown:FunctionalTyp 6.134928e+04
NeighborhoodSawyer:FunctionalTyp 4.287953e+04
NeighborhoodSawyerW:FunctionalTyp -2.096883e+04
NeighborhoodSomerst:FunctionalTyp -3.785206e+04
NeighborhoodStoneBr:FunctionalTyp -3.541593e+04
NeighborhoodSWISU:FunctionalTyp 3.466531e+04
NeighborhoodTimber:FunctionalTyp -3.372127e+04
NeighborhoodVeenker:FunctionalTyp 1.043751e+05
NeighborhoodBlueste:Fireplaces -4.303970e+04
NeighborhoodBrDale:Fireplaces -5.406090e+03
NeighborhoodBrkSide:Fireplaces -8.813769e+03
NeighborhoodClearCr:Fireplaces -4.054900e+03
NeighborhoodCollgCr:Fireplaces -2.087977e+03
NeighborhoodCrawfor:Fireplaces 1.177167e+03
NeighborhoodEdwards:Fireplaces 3.554600e+03
NeighborhoodGilbert:Fireplaces -2.856974e+03
NeighborhoodGreens:Fireplaces 1.284140e+04
NeighborhoodGrnHill:Fireplaces .
NeighborhoodIDOTRR:Fireplaces 1.820482e+03
NeighborhoodLandmrk:Fireplaces .
NeighborhoodMeadowV:Fireplaces -1.093472e+04
NeighborhoodMitchel:Fireplaces -6.468019e+02
NeighborhoodNAmes:Fireplaces -6.018056e+03
NeighborhoodNoRidge:Fireplaces -1.151900e+04
NeighborhoodNPkVill:Fireplaces -4.061900e+04
NeighborhoodNridgHt:Fireplaces 2.193598e+04
NeighborhoodNWAmes:Fireplaces -1.118831e+04
NeighborhoodOldTown:Fireplaces -5.539852e+03
NeighborhoodSawyer:Fireplaces -5.016812e+03
NeighborhoodSawyerW:Fireplaces 1.334180e+03
NeighborhoodSomerst:Fireplaces 8.228491e+03
NeighborhoodStoneBr:Fireplaces 1.737667e+02
NeighborhoodSWISU:Fireplaces -7.926273e+03
NeighborhoodTimber:Fireplaces 8.246769e+03
NeighborhoodVeenker:Fireplaces -2.377451e+04
NeighborhoodBlueste:Garage.Cars 1.879425e+04
NeighborhoodBrDale:Garage.Cars 3.878644e+03
NeighborhoodBrkSide:Garage.Cars 2.433543e+04
NeighborhoodClearCr:Garage.Cars 1.913935e+04
NeighborhoodCollgCr:Garage.Cars 2.467441e+04
NeighborhoodCrawfor:Garage.Cars 1.056673e+04
NeighborhoodEdwards:Garage.Cars 1.495691e+04
NeighborhoodGilbert:Garage.Cars 2.958493e+04
NeighborhoodGreens:Garage.Cars .
NeighborhoodGrnHill:Garage.Cars .
NeighborhoodIDOTRR:Garage.Cars 1.746818e+04
NeighborhoodLandmrk:Garage.Cars .
NeighborhoodMeadowV:Garage.Cars 1.933489e+04
NeighborhoodMitchel:Garage.Cars 1.871638e+04
NeighborhoodNAmes:Garage.Cars 1.750737e+04
NeighborhoodNoRidge:Garage.Cars 2.495561e+04
NeighborhoodNPkVill:Garage.Cars 3.349815e+03
NeighborhoodNridgHt:Garage.Cars 3.007006e+04
NeighborhoodNWAmes:Garage.Cars 1.524504e+04
NeighborhoodOldTown:Garage.Cars 1.519914e+04
NeighborhoodSawyer:Garage.Cars 1.579902e+04
NeighborhoodSawyerW:Garage.Cars 9.375270e+03
NeighborhoodSomerst:Garage.Cars 2.958286e+03
NeighborhoodStoneBr:Garage.Cars 4.022816e+04
NeighborhoodSWISU:Garage.Cars 7.792872e+03
NeighborhoodTimber:Garage.Cars 1.840704e+04
NeighborhoodVeenker:Garage.Cars 1.764819e+04
NeighborhoodBlueste:Paved.DriveP .
NeighborhoodBrDale:Paved.DriveP .
NeighborhoodBrkSide:Paved.DriveP 6.960465e+03
NeighborhoodClearCr:Paved.DriveP .
NeighborhoodCollgCr:Paved.DriveP .
NeighborhoodCrawfor:Paved.DriveP 7.966641e+03
NeighborhoodEdwards:Paved.DriveP 2.965882e+04
NeighborhoodGilbert:Paved.DriveP .
NeighborhoodGreens:Paved.DriveP .
NeighborhoodGrnHill:Paved.DriveP .
NeighborhoodIDOTRR:Paved.DriveP 6.657340e+02
NeighborhoodLandmrk:Paved.DriveP .
NeighborhoodMeadowV:Paved.DriveP .
NeighborhoodMitchel:Paved.DriveP 1.064167e+04
NeighborhoodNAmes:Paved.DriveP .
NeighborhoodNoRidge:Paved.DriveP .
NeighborhoodNPkVill:Paved.DriveP .
NeighborhoodNridgHt:Paved.DriveP .
NeighborhoodNWAmes:Paved.DriveP .
NeighborhoodOldTown:Paved.DriveP 6.159125e+02
NeighborhoodSawyer:Paved.DriveP .
NeighborhoodSawyerW:Paved.DriveP .
NeighborhoodSomerst:Paved.DriveP .
NeighborhoodStoneBr:Paved.DriveP .
NeighborhoodSWISU:Paved.DriveP 4.117996e+04
NeighborhoodTimber:Paved.DriveP .
NeighborhoodVeenker:Paved.DriveP .
NeighborhoodBlueste:Paved.DriveY 3.215277e+04
NeighborhoodBrDale:Paved.DriveY 1.284032e+04
NeighborhoodBrkSide:Paved.DriveY -1.391058e+03
NeighborhoodClearCr:Paved.DriveY 6.993697e+03
NeighborhoodCollgCr:Paved.DriveY -1.192901e+04
NeighborhoodCrawfor:Paved.DriveY 2.691864e+04
NeighborhoodEdwards:Paved.DriveY 1.984422e+04
NeighborhoodGilbert:Paved.DriveY -2.135256e+04
NeighborhoodGreens:Paved.DriveY .
NeighborhoodGrnHill:Paved.DriveY 2.355415e+03
NeighborhoodIDOTRR:Paved.DriveY 1.554200e+03
NeighborhoodLandmrk:Paved.DriveY .
NeighborhoodMeadowV:Paved.DriveY .
NeighborhoodMitchel:Paved.DriveY 6.896190e+01
NeighborhoodNAmes:Paved.DriveY 1.606338e+04
NeighborhoodNoRidge:Paved.DriveY -1.604663e+04
NeighborhoodNPkVill:Paved.DriveY 4.393436e+04
NeighborhoodNridgHt:Paved.DriveY -6.663016e+04
NeighborhoodNWAmes:Paved.DriveY -3.067297e+04
NeighborhoodOldTown:Paved.DriveY 6.233042e+03
NeighborhoodSawyer:Paved.DriveY 4.592254e+03
NeighborhoodSawyerW:Paved.DriveY 4.834718e+04
NeighborhoodSomerst:Paved.DriveY 1.468738e+03
NeighborhoodStoneBr:Paved.DriveY -5.827632e+04
NeighborhoodSWISU:Paved.DriveY 4.429564e+04
NeighborhoodTimber:Paved.DriveY -1.705226e+04
NeighborhoodVeenker:Paved.DriveY -4.510289e+04
Condition.1Artery:Overall.Qual 2.351656e+03
Condition.1Feedr:Overall.Qual -4.107351e+03
Condition.1Park:Overall.Qual 8.747319e+02
Condition.1Rail:Overall.Qual 1.577419e+03
Condition.1Artery:Overall.Cond 2.298631e+03
Condition.1Feedr:Overall.Cond 1.701441e+03
Condition.1Park:Overall.Cond 3.508284e+03
Condition.1Rail:Overall.Cond -1.316311e+04
Condition.1Artery:HouseAge 1.719589e+02
Condition.1Feedr:HouseAge -4.364914e+01
Condition.1Park:HouseAge 4.424944e+01
Condition.1Rail:HouseAge -7.091646e+02
Condition.1Artery:FoundationCBlock 7.311144e+03
Condition.1Feedr:FoundationCBlock 8.927356e+03
Condition.1Park:FoundationCBlock -3.511680e+04
Condition.1Rail:FoundationCBlock 1.226001e+04
Condition.1Artery:FoundationPConc 1.165727e+04
Condition.1Feedr:FoundationPConc 3.362105e+03
Condition.1Park:FoundationPConc -3.218194e+04
Condition.1Rail:FoundationPConc 3.939639e+04
Condition.1Artery:FoundationSlab .
Condition.1Feedr:FoundationSlab 2.086550e+04
Condition.1Park:FoundationSlab -2.610248e+04
Condition.1Rail:FoundationSlab .
Condition.1Artery:FoundationStone .
Condition.1Feedr:FoundationStone .
Condition.1Park:FoundationStone .
Condition.1Rail:FoundationStone .
Condition.1Artery:FoundationWood .
Condition.1Feedr:FoundationWood .
Condition.1Park:FoundationWood .
Condition.1Rail:FoundationWood .
Condition.1Artery:Total.Bsmt.SF 3.067522e+01
Condition.1Feedr:Total.Bsmt.SF 2.278623e+01
Condition.1Park:Total.Bsmt.SF 1.003780e+01
Condition.1Rail:Total.Bsmt.SF -8.616873e+00
Condition.1Artery:Central.AirY 2.008637e+03
Condition.1Feedr:Central.AirY 5.238952e+02
Condition.1Park:Central.AirY -1.816071e+04
Condition.1Rail:Central.AirY 5.077456e+03
Condition.1Artery:log(X12.SF) -8.479033e+03
Condition.1Feedr:log(X12.SF) -5.340168e+03
Condition.1Park:log(X12.SF) 6.311889e+03
Condition.1Rail:log(X12.SF) 7.236432e+03
Condition.1Artery:Baths -2.724680e+02
Condition.1Feedr:Baths -8.790418e+01
Condition.1Park:Baths -5.141049e+03
Condition.1Rail:Baths -6.496596e+03
Condition.1Artery:Kitchen.AbvGr 3.360335e+02
Condition.1Feedr:Kitchen.AbvGr -4.677704e+03
Condition.1Park:Kitchen.AbvGr -3.921464e+04
Condition.1Rail:Kitchen.AbvGr 6.348843e+04
Condition.1Artery:Kitchen.Qual -1.062697e+03
Condition.1Feedr:Kitchen.Qual -4.183542e+03
Condition.1Park:Kitchen.Qual 4.557790e+03
Condition.1Rail:Kitchen.Qual -9.075642e+03
Condition.1Artery:FunctionalMaj2 .
Condition.1Feedr:FunctionalMaj2 .
Condition.1Park:FunctionalMaj2 .
Condition.1Rail:FunctionalMaj2 .
Condition.1Artery:FunctionalMin1 .
Condition.1Feedr:FunctionalMin1 -1.316042e+03
Condition.1Park:FunctionalMin1 -1.216045e+05
Condition.1Rail:FunctionalMin1 6.237964e+04
Condition.1Artery:FunctionalMin2 .
Condition.1Feedr:FunctionalMin2 1.019535e+05
Condition.1Park:FunctionalMin2 -9.902724e+03
Condition.1Rail:FunctionalMin2 5.112224e+04
Condition.1Artery:FunctionalMod -2.491374e+04
Condition.1Feedr:FunctionalMod 4.215363e+04
Condition.1Park:FunctionalMod -6.379925e+04
Condition.1Rail:FunctionalMod .
Condition.1Artery:FunctionalTyp -4.435788e+03
Condition.1Feedr:FunctionalTyp 3.851065e+04
Condition.1Park:FunctionalTyp -2.570331e+03
Condition.1Rail:FunctionalTyp 6.733929e+04
Condition.1Artery:Fireplaces -4.121435e+03
Condition.1Feedr:Fireplaces -4.367961e+03
Condition.1Park:Fireplaces -9.981571e+03
Condition.1Rail:Fireplaces -5.440808e+03
Condition.1Artery:Garage.Cars 8.416206e+02
Condition.1Feedr:Garage.Cars -7.429902e+01
Condition.1Park:Garage.Cars -5.794487e+03
Condition.1Rail:Garage.Cars -2.175790e+04
Condition.1Artery:Paved.DriveP -4.904995e+04
Condition.1Feedr:Paved.DriveP -1.291059e+04
Condition.1Park:Paved.DriveP .
Condition.1Rail:Paved.DriveP .
Condition.1Artery:Paved.DriveY -1.937015e+04
Condition.1Feedr:Paved.DriveY -5.171342e+03
Condition.1Park:Paved.DriveY .
Condition.1Rail:Paved.DriveY -1.218357e+04
Overall.Qual:Overall.Cond 5.824994e+02
Overall.Qual:HouseAge 4.192631e+00
Overall.Qual:FoundationCBlock 1.800083e+03
Overall.Qual:FoundationPConc -1.150747e+03
Overall.Qual:FoundationSlab 7.238879e+03
Overall.Qual:FoundationStone 9.019059e+02
Overall.Qual:FoundationWood .
Overall.Qual:Total.Bsmt.SF 1.028472e+01
Overall.Qual:Central.AirY -3.763103e+02
Overall.Qual:log(X12.SF) 1.457402e+03
Overall.Qual:Baths 3.018332e+03
Overall.Qual:Kitchen.AbvGr -1.145279e+04
Overall.Qual:Kitchen.Qual 9.746687e+02
Overall.Qual:FunctionalMaj2 1.292167e+03
Overall.Qual:FunctionalMin1 -1.025699e+04
Overall.Qual:FunctionalMin2 -1.039224e+04
Overall.Qual:FunctionalMod .
Overall.Qual:FunctionalTyp -1.136515e+04
Overall.Qual:Fireplaces 4.013016e+02
Overall.Qual:Garage.Cars 2.219854e+03
Overall.Qual:Paved.DriveP -3.952631e+03
Overall.Qual:Paved.DriveY -6.072074e+03
Overall.Cond:HouseAge 5.169968e+01
Overall.Cond:FoundationCBlock -2.811838e+03
Overall.Cond:FoundationPConc -1.841508e+03
Overall.Cond:FoundationSlab 4.795250e+03
Overall.Cond:FoundationStone .
Overall.Cond:FoundationWood -3.942050e+03
Overall.Cond:Total.Bsmt.SF 3.769156e+00
Overall.Cond:Central.AirY -3.003502e+03
Overall.Cond:log(X12.SF) -1.032565e+03
Overall.Cond:Baths -5.497586e+02
Overall.Cond:Kitchen.AbvGr 8.738990e+03
Overall.Cond:Kitchen.Qual 7.617683e+02
Overall.Cond:FunctionalMaj2 .
Overall.Cond:FunctionalMin1 1.128087e+03
Overall.Cond:FunctionalMin2 1.111867e+04
Overall.Cond:FunctionalMod 1.203638e+04
Overall.Cond:FunctionalTyp 1.052922e+04
Overall.Cond:Fireplaces 1.356432e+03
Overall.Cond:Garage.Cars -1.863690e+02
Overall.Cond:Paved.DriveP 5.656957e+03
Overall.Cond:Paved.DriveY 3.370411e+03
HouseAge:FoundationCBlock -1.182097e+02
HouseAge:FoundationPConc -4.074844e+02
HouseAge:FoundationSlab 3.469731e+02
HouseAge:FoundationStone .
HouseAge:FoundationWood 2.817590e+03
HouseAge:Total.Bsmt.SF -9.247162e-02
HouseAge:Central.AirY 2.637132e+02
HouseAge:log(X12.SF) -6.219200e+01
HouseAge:Baths -7.023008e+01
HouseAge:Kitchen.AbvGr -2.126380e+02
HouseAge:Kitchen.Qual 6.185930e+01
HouseAge:FunctionalMaj2 .
HouseAge:FunctionalMin1 4.661907e+01
HouseAge:FunctionalMin2 2.051704e+02
HouseAge:FunctionalMod -4.016333e+02
HouseAge:FunctionalTyp -6.113925e+00
HouseAge:Fireplaces 1.846116e+01
HouseAge:Garage.Cars 5.747629e+01
HouseAge:Paved.DriveP 2.460558e+02
HouseAge:Paved.DriveY 8.509803e+01
FoundationCBlock:Total.Bsmt.SF -1.434188e+01
FoundationPConc:Total.Bsmt.SF -1.493536e+01
FoundationSlab:Total.Bsmt.SF .
FoundationStone:Total.Bsmt.SF 1.337291e-03
FoundationWood:Total.Bsmt.SF -1.803278e+01
FoundationCBlock:Central.AirY 1.813977e+03
FoundationPConc:Central.AirY 1.120615e+04
FoundationSlab:Central.AirY -1.117415e+03
FoundationStone:Central.AirY 1.438219e+04
FoundationWood:Central.AirY -1.136530e+03
FoundationCBlock:log(X12.SF) -2.745992e+03
FoundationPConc:log(X12.SF) 3.079723e+03
FoundationSlab:log(X12.SF) .
FoundationStone:log(X12.SF) .
FoundationWood:log(X12.SF) .
FoundationCBlock:Baths -1.380775e+03
FoundationPConc:Baths -4.688492e+03
FoundationSlab:Baths 1.877183e+04
FoundationStone:Baths .
FoundationWood:Baths -2.908866e+03
FoundationCBlock:Kitchen.AbvGr 4.762548e+03
FoundationPConc:Kitchen.AbvGr 2.511972e+04
FoundationSlab:Kitchen.AbvGr -4.321421e+04
FoundationStone:Kitchen.AbvGr -1.272784e+03
FoundationWood:Kitchen.AbvGr .
FoundationCBlock:Kitchen.Qual 1.348843e+03
FoundationPConc:Kitchen.Qual -2.801662e+03
FoundationSlab:Kitchen.Qual -4.243940e+03
FoundationStone:Kitchen.Qual .
FoundationWood:Kitchen.Qual -1.553436e+03
FoundationCBlock:FunctionalMaj2 .
FoundationPConc:FunctionalMaj2 .
FoundationSlab:FunctionalMaj2 .
FoundationStone:FunctionalMaj2 .
FoundationWood:FunctionalMaj2 .
FoundationCBlock:FunctionalMin1 7.498815e+04
FoundationPConc:FunctionalMin1 5.454084e+04
FoundationSlab:FunctionalMin1 1.666718e+05
FoundationStone:FunctionalMin1 .
FoundationWood:FunctionalMin1 .
FoundationCBlock:FunctionalMin2 -1.302539e+04
FoundationPConc:FunctionalMin2 -3.421082e+04
FoundationSlab:FunctionalMin2 3.799190e+04
FoundationStone:FunctionalMin2 4.844266e+04
FoundationWood:FunctionalMin2 .
FoundationCBlock:FunctionalMod -1.512480e+04
FoundationPConc:FunctionalMod -8.024713e+04
FoundationSlab:FunctionalMod -1.000125e+04
FoundationStone:FunctionalMod .
FoundationWood:FunctionalMod .
FoundationCBlock:FunctionalTyp 3.516194e+01
FoundationPConc:FunctionalTyp -2.679816e+04
FoundationSlab:FunctionalTyp 5.383759e+04
FoundationStone:FunctionalTyp -2.743973e+04
FoundationWood:FunctionalTyp .
FoundationCBlock:Fireplaces 4.199150e+02
FoundationPConc:Fireplaces 2.394544e+03
FoundationSlab:Fireplaces 1.697975e+04
FoundationStone:Fireplaces .
FoundationWood:Fireplaces .
FoundationCBlock:Garage.Cars 1.683153e+03
FoundationPConc:Garage.Cars 6.799961e+02
FoundationSlab:Garage.Cars -8.313840e+03
FoundationStone:Garage.Cars .
FoundationWood:Garage.Cars .
FoundationCBlock:Paved.DriveP -4.517045e+03
FoundationPConc:Paved.DriveP -3.192248e+03
FoundationSlab:Paved.DriveP .
FoundationStone:Paved.DriveP .
FoundationWood:Paved.DriveP .
FoundationCBlock:Paved.DriveY 6.053565e+03
FoundationPConc:Paved.DriveY 8.751293e+03
FoundationSlab:Paved.DriveY -3.104708e+04
FoundationStone:Paved.DriveY .
FoundationWood:Paved.DriveY .
Total.Bsmt.SF:Central.AirY -1.757439e+01
Total.Bsmt.SF:log(X12.SF) -4.125011e+00
Total.Bsmt.SF:Baths 1.167554e+01
Total.Bsmt.SF:Kitchen.AbvGr -2.549120e+01
Total.Bsmt.SF:Kitchen.Qual 7.871198e-01
Total.Bsmt.SF:FunctionalMaj2 -2.859242e+01
Total.Bsmt.SF:FunctionalMin1 -1.685976e+01
Total.Bsmt.SF:FunctionalMin2 -2.520450e+01
Total.Bsmt.SF:FunctionalMod -6.850826e+01
Total.Bsmt.SF:FunctionalTyp -3.544242e+01
Total.Bsmt.SF:Fireplaces 2.491681e+00
Total.Bsmt.SF:Garage.Cars -5.130345e+00
Total.Bsmt.SF:Paved.DriveP -1.595442e+01
Total.Bsmt.SF:Paved.DriveY 1.518944e+00
Central.AirY:log(X12.SF) 1.369315e+03
Central.AirY:Baths -2.410431e+03
Central.AirY:Kitchen.AbvGr 1.451956e+04
Central.AirY:Kitchen.Qual 6.651701e+03
Central.AirY:FunctionalMaj2 9.314278e+03
Central.AirY:FunctionalMin1 .
Central.AirY:FunctionalMin2 2.197726e+04
Central.AirY:FunctionalMod 5.992600e+04
Central.AirY:FunctionalTyp 5.616262e+04
Central.AirY:Fireplaces -5.524425e+03
Central.AirY:Garage.Cars -5.513441e+02
Central.AirY:Paved.DriveP 1.888029e+04
Central.AirY:Paved.DriveY 1.180651e+04
log(X12.SF):Baths 3.742547e+03
log(X12.SF):Kitchen.AbvGr -3.526215e+03
log(X12.SF):Kitchen.Qual -9.314317e+02
log(X12.SF):FunctionalMaj2 9.523529e+03
log(X12.SF):FunctionalMin1 -2.911798e+03
log(X12.SF):FunctionalMin2 9.829218e+02
log(X12.SF):FunctionalMod 1.998501e+03
log(X12.SF):FunctionalTyp 1.161523e+02
log(X12.SF):Fireplaces 1.554477e+03
log(X12.SF):Garage.Cars -1.430209e+03
log(X12.SF):Paved.DriveP 1.580954e+02
log(X12.SF):Paved.DriveY 2.550142e+03
Baths:Kitchen.AbvGr -5.982323e+03
Baths:Kitchen.Qual 3.782446e+02
Baths:FunctionalMaj2 .
Baths:FunctionalMin1 -1.761324e+04
Baths:FunctionalMin2 -2.030803e+04
Baths:FunctionalMod -2.186861e+04
Baths:FunctionalTyp -2.675980e+04
Baths:Fireplaces -4.663751e+03
Baths:Garage.Cars 2.040156e+03
Baths:Paved.DriveP 4.924447e+02
Baths:Paved.DriveY -9.064070e+02
Kitchen.AbvGr:Kitchen.Qual -1.007154e+04
Kitchen.AbvGr:FunctionalMaj2 .
Kitchen.AbvGr:FunctionalMin1 1.683451e+04
Kitchen.AbvGr:FunctionalMin2 -9.872511e+03
Kitchen.AbvGr:FunctionalMod 6.814211e+04
Kitchen.AbvGr:FunctionalTyp 6.592839e+04
Kitchen.AbvGr:Fireplaces -1.686283e+00
Kitchen.AbvGr:Garage.Cars 2.600280e+03
Kitchen.AbvGr:Paved.DriveP 1.143468e+04
Kitchen.AbvGr:Paved.DriveY 1.344158e+04
Kitchen.Qual:FunctionalMaj2 9.348984e+02
Kitchen.Qual:FunctionalMin1 .
Kitchen.Qual:FunctionalMin2 -7.266111e+03
Kitchen.Qual:FunctionalMod .
Kitchen.Qual:FunctionalTyp 9.218779e+02
Kitchen.Qual:Fireplaces 1.266550e+03
Kitchen.Qual:Garage.Cars 1.718522e+03
Kitchen.Qual:Paved.DriveP .
Kitchen.Qual:Paved.DriveY -5.191888e+03
FunctionalMaj2:Fireplaces -5.996386e+04
FunctionalMin1:Fireplaces -1.600998e+04
FunctionalMin2:Fireplaces -1.775667e+04
FunctionalMod:Fireplaces -1.957023e+00
FunctionalTyp:Fireplaces -1.925056e+04
FunctionalMaj2:Garage.Cars .
FunctionalMin1:Garage.Cars -2.172384e+03
FunctionalMin2:Garage.Cars 2.463296e+03
FunctionalMod:Garage.Cars .
FunctionalTyp:Garage.Cars -6.756674e+03
FunctionalMaj2:Paved.DriveP .
FunctionalMin1:Paved.DriveP .
FunctionalMin2:Paved.DriveP 4.215951e+03
FunctionalMod:Paved.DriveP -3.261451e+04
FunctionalTyp:Paved.DriveP 4.508108e+03
FunctionalMaj2:Paved.DriveY .
FunctionalMin1:Paved.DriveY -4.465952e+04
FunctionalMin2:Paved.DriveY -1.222873e+04
FunctionalMod:Paved.DriveY -2.954323e+04
FunctionalTyp:Paved.DriveY 2.005576e+00
Fireplaces:Garage.Cars 1.111989e+03
Fireplaces:Paved.DriveP 1.242086e+04
Fireplaces:Paved.DriveY 1.161963e+04
Garage.Cars:Paved.DriveP 6.117077e+02
Garage.Cars:Paved.DriveY -2.528726e+03
Bsmt.YN:Bsmt.ExposureNone .
Bsmt.YN:Bsmt.Exposure -4.268130e+03
Bsmt.YN:Bsmt.ExposureAv 2.281787e+04
Bsmt.YN:Bsmt.ExposureGd 3.010838e+04
Bsmt.YN:Bsmt.ExposureMn 1.791112e+04
Bsmt.YN:Bsmt.ExposureNo 1.849160e+04housing.blasso.RJ = blasso(X.scaled, Y, RJ = TRUE, verb=0)
Error in blasso(X.scaled, Y, RJ = TRUE, verb = 0) :
NA/NaN/Inf in foreign function call (arg 5)# center and scale X
X.scaled = X
X.scaled[,2:15] = scale(X[,2:15], center =college.ridge.cv$xm[c(-1,-16)],
scale = college.ridge.cv$scales[c(-1,-16)])
# RJ = FALSE
# college.blasso = blasso(X.scaled, Y, RJ = FALSE, verb=0)
# ypred.blasso = mean(college.blasso$mu) +
# as.matrix(test.scaled) %*% apply(college.blasso$beta, 2, mean)
# rmse.blasso = rmse(exp(ctest$Apps), exp(ypred.blasso))
# model selection with RJ = TRUE
college.blasso.RJ = blasso(X.scaled, Y, RJ = TRUE, verb=0)
ypred.blasso.RJ = mean(college.blasso.RJ$mu) + as.matrix(test.scaled) %*% apply(college.blasso.RJ$beta, 2, mean)
rmse.blasso.RJ = rmse(exp(ctest$Apps), exp(ypred.blasso.RJ))
summary(college.blasso.RJ)$coefmodel.lasso = glmnet(X.train, log(data_train$price), alpha=1)
cv.lasso = cv.glmnet(X.train, log(data_train$price), alpha=1)
yhat.lasso = exp(predict(model.lasso, s=cv.lasso$lambda.min, type="response", newx = X.test))
sqrt(mean(((yhat.lasso)-data_test$price)^2))
# Ridge for the simplified model (from BIC)
model.ridge = glmnet(X.train, log(data_train$price), alpha=0)
cv.ridge = cv.glmnet(X.train, log(data_train$price), alpha=0)
yhat.ridge = predict(model.ridge, s=cv.ridge$lambda.min, type="response", newx = X.test)
sqrt(mean((exp(yhat.ridge)-data_test$price)^2))Part III: Write Up
Once you are satisfied with your model, provide a write up of your data analysis project in a new Rmd file/pdf file: writeup.Rmd by copying over salient parts of your R notebook. The written assignment consists of five parts:
Exploratory data analysis (20 points): must include three correctly labeled graphs and an explanation that highlight the most important features that went into your model building.
Development and assessment of an initial model from Part I (10 points)
Initial model: must include a summary table and an explanation/discussion for variable selection. Interpretation of coefficients desirable for full points.
Model selection: must include a discussion
Residual: must include a residual plot and a discussion
RMSE: must include an RMSE and an explanation (other criteria desirable)
Model testing: must include an explanation
- Development of the final model (20 points)
Final model: must include a summary table
Variables: must include an explanation
Variable selection/shrinkage: must use appropriate method and include an explanation
- Assessment of the final model (25 points)
Residual: must include a residual plot and a discussion
RMSE: must include an RMSE and an explanation (other criteria desirable)
Model evaluation: must include an evaluation discussion
Model testing : must include a discussion
Model result: must include a selection of the top 10 undervalued and overvalued houses
- Conclusion (10 points): must include a summary of results and a discussion of things learned
Part IV
Create predictions for the validation data from your final model and write out to a file prediction-validation.Rdata This should have the same format as the models in Part I and II.
10 points
Class Presentations
Each Group should prepare 5 slides in their Github repo: (save as slides.pdf)
Most interesting graphic (a picture is worth a thousand words prize!)
Best Model (motivation, how you found it, why you think it is best)
Best Insights into predicting Sales Price.
2 Best Houses to purchase (and why)
Best Team Name/Graphic
We will select winners based on the above criteria and overall performance.
Finally your repo should have: writeup.Rmd, writeup.pdf, slides.Rmd (and whatever output you use for the presentation) and predict.Rdata and predict-validation.Rdata.
LS0tCnRpdGxlOiAiRmluYWwgRGF0YSBBbmFseXNpcyBQcm9qZWN0IgpkYXRlOiAgIldyaXRlIHVwIGR1ZSBBcHJpbCAyOCBhdCA1IHBtIgpvdXRwdXQ6IGh0bWxfbm90ZWJvb2sKLS0tCgpgYGB7ciBzZXR1cCwgaW5jbHVkZT1GQUxTRX0Ka25pdHI6Om9wdHNfY2h1bmskc2V0KGVjaG8gPSBUUlVFKQpgYGAKCgpGb3IgdGhpcyBwcm9qZWN0IHlvdSB3aWxsIHRha2UgdGhlIHJvbGUgb2YgYSBjb25zdWx0YW50IGhpcmVkIGJ5IGEgcmVhbCBlc3RhdGUgaW52ZXN0bWVudCBmaXJtIGluIEFtZXMsIElvd2EsIGEgbWlkLXdlc3QgdG93biBpbiB0aGUgVW5pdGVkIFN0YXRlcywgdG8gYW5hbHl6ZSBkYXRhIGluIG9yZGVyIHRvIGhlbHAgcHJvdmlkZSBpbnNpZ2h0IGludG8gaG93IHRoZSBmaXJtIHNob3VsZCBpbnZlc3QgZm9yIGhpZ2hlc3QgcHJvZml0cywgYW5kIHRvIHF1YW50aWZ5IGFuZCBjb21tdW5pY2F0ZSB0byB0aGUgY29tcGFueSBtYW5hZ2VtZW50IHdoYXQgdHlwZXMgb2YgcmVhbCBlc3RhdGUgcHJvcGVydGllcyBhcmUgZ29vZCBpbnZlc3RtZW50cyBhbmQgd2h5LiBUaGV5IGhhdmUgcHJvdmlkZWQgeW91IHdpdGggZGF0YSBvbiBob3VzaW5nIHNhbGVzIGZyb20gYmV0d2VlbiAyMDA2IHRvIDIwMTAgdGhhdCBjb250YWlucyBpbmZvcm1hdGlvbiBhYm91dCB0aGUgY2hhcmFjdGVyaXN0aWNzIG9mIHRoZSBob3VzZSAobnVtYmVyIG9mIGJlZHJvb21zLCBudW1iZXIgb2YgYmF0aHJvb21zLCBzcXVhcmUgZm9vdGFnZSwgZXRjLikgYW5kIHRoZSBob3VzZSdzIHNhbGUgcHJpY2UuIFRoZSBjb2RlYm9vayBmb3IgdGhpcyBkYXRhIHNldCBpcyBbYXZhaWxhYmxlIG9ubGluZSBoZXJlIF0oaHR0cHM6Ly93dzIuYW1zdGF0Lm9yZy9wdWJsaWNhdGlvbnMvanNlL3YxOW4zL2RlY29jay9kYXRhZG9jdW1lbnRhdGlvbi50eHQpICBvciBpbiB0aGUgRGF0YSBmb2xkZXIgaW4geW91ciByZXBvLgoKIyMgQWJvdXQgdGhlIERhdGEgQW5hbHlzaXMgUHJvamVjdAoKSXQncyBnZW5lcmFsbHkgYSBiYWQgaWRlYSB0byBidXkgdGhlIG1vc3QgZXhwZW5zaXZlIGhvdXNlIGluIHRoZSBuZWlnaGJvcmhvb2QuIEFuZCByZW1lbWJlciB0aGUgcmVhbCBlc3RhdGUgYWdlbnRzJyBtYW50cmE6IExvY2F0aW9uLCBsb2NhdGlvbiwgbG9jYXRpb24hIEtlZXAgaW4gbWluZCB0aGF0IHRoZSBnb2FsIGlzIHRvIG1ha2UgbW9uZXkgZm9yIHlvdXIgaW52ZXN0b3JzLCBhbmQgaGVuY2UgaW52ZXN0aW5nIGluIGEgcHJvcGVydHkgdGhhdCBpcyBvdmVydmFsdWVkIChjb3N0aW5nIG1vcmUgdGhhbiBpdCBpcyB3b3J0aCkgaXMgcmFyZWx5IGEgZ29vZCBpZGVhLiBUaGlzIG1lYW5zIHRoYXQgaXQncyBjcml0aWNhbCB0byBrbm93IHdoaWNoIHByb3BlcnRpZXMgYXJlIG92ZXJ2YWx1ZWQgYW5kIHdoaWNoIGFyZSB1bmRlcnZhbHVlZC4gIFRoZSBjb21wYW55IHRoYXQgaGlyZWQgeW91IGhhcyBtYW55IHF1ZXN0aW9ucyBmb3IgeW91IGFib3V0IHRoZSBob3VzaW5nIG1hcmtldC4gSXQgaXMgdXAgdG8geW91IHRvIGRlY2lkZSB3aGF0IG1ldGhvZHMgeW91IHdhbnQgdG8gdXNlIChmcmVxdWVudGlzdCBvciBCYXllc2lhbikgdG8gYW5zd2VyIHRoZXNlIHF1ZXN0aW9ucywgYW5kIGltcGxlbWVudCB0aGVtIHRvIGhlbHAgdG8gaWRlbnRpZnkgdW5kZXJ2YWx1ZWQgYW5kIG92ZXJ2YWx1ZWQgcHJvcGVydGllcy4KCgpZb3Ugd2lsbCBoYXZlIHRocmVlIGRhdGEgc2V0czogYSBzdWJzZXQgZm9yIHRyYWluaW5nLCBhIHN1YnNldCBmb3IgdGVzdGluZywgYW5kIGEgdGhpcmQgc3Vic2V0IGZvciB2YWxpZGF0aW9uLiBZb3Ugd2lsbCBiZSBhc2tlZCB0byBkbyBkYXRhIGV4cGxvcmF0aW9uIGFuZCBidWlsZCB5b3VyIG1vZGVsIChvciBtb2RlbHMpIGluaXRpYWxseSB1c2luZyBvbmx5IHRoZSB0cmFpbmluZyBkYXRhLiBUaGVuLCB5b3Ugd2lsbCB0ZXN0IHlvdXIgbW9kZWwgb24gdGhlIHRlc3RpbmcgZGF0YSwgYW5kIGZpbmFsbHkgdmFsaWRhdGUgdXNpbmcgdGhlIHZhbGlkYXRpb24gZGF0YS4gV2UgYXJlIGNoYWxsZW5naW5nIHlvdSB0byBrZWVwIHlvdXIgYW5hbHlzaXMgZXhwZXJpZW5jZSByZWFsaXN0aWMsIGFuZCBpbiBhIHJlYWxpc3RpYyBzY2VuYXJpbyB5b3Ugd291bGQgbm90IGhhdmUgYWNjZXNzIHRvIGFsbCB0aHJlZSBvZiB0aGVzZSBkYXRhIHNldHMgYXQgb25jZS4gIFlvdSB3aWxsIGJlIGFibGUgdG8gc2VlIG9uIG91ciBzY29yZWJvYXJkIGhvdyB3ZWxsIHlvdXIgdGVhbSBpcyBkb2luZyBiYXNlZCBvbiBpdHMgcHJlZGljdGl2ZSBwZXJmb3JtYW5jZSBvbiB0aGUgdGVzdGluZyBkYXRhLiAgQWZ0ZXIgeW91ciBwcm9qZWN0IGlzIHR1cm5lZCBpbiB5b3Ugd2lsbCBzZWUgdGhlIGZpbmFsIHNjb3JlIG9uIHRoZSB2YWxpZGF0aW9uIHNldC4KCkFsbCBtZW1iZXJzIG9mIHRoZSB0ZWFtIHNob3VsZCBjb250cmlidXRlIGVxdWFsbHkgYW5kIGFuc3dlciBhbnkgcXVlc3Rpb25zIGFib3V0IHRoZSBhbmFseXNpcyBhdCB0aGUgZmluYWwgcHJlc2VudGF0aW9uLgoKRm9yIHlvdXIgYW5hbHlzaXMgY3JlYXRlIGEgbmV3IG5vdGVib29rIG5hbWVkICJwcm9qZWN0LlJtZCIKYW5kIHVwZGF0ZSBhY2NvcmRpbmdseSByYXRoZXIgdGhhbiBlZGl0aW5nIHRoaXMuCgoKIyMjIFJlYWQgaW4gVHJhaW5pbmcgRGF0YQoKVG8gZ2V0IHN0YXJ0ZWQgcmVhZCBpbiB0aGUgdHJhaW5pbmcgZGF0YS4gCmBgYHtyIHJlYWQtZGF0YX0KbGlicmFyeShkcGx5cikKbGlicmFyeSh0aWR5cikKbG9hZCgiYW1lc190cmFpbi5SZGF0YSIpCgpwcmludChwYXN0ZTAoIlRoZSBkYXRhc2V0IGhhcyAiLCBkaW0oYW1lc190cmFpbilbMV0sICIgbnVtYmVyIG9mIG9ic2VydmF0aW9ucyBhbmQgIiwgZGltKGFtZXNfdHJhaW4pWzJdLCAiIGZlYXR1cmVzIikpCgojVmFyaWFibGVzIHdpdGggTkEncyBhbmQgdGhlaXIgcHJvcG9ydGlvbiBvZiBtaXNzaW5nIGRhdGEKbWlzcyA9IGFwcGx5KGlzLm5hKGFtZXNfdHJhaW4pLCAyLCBzdW0pCm1pc3NfcHJvcCA9IHJvdW5kKG1pc3NbbWlzcz4wXS9ucm93KGFtZXNfdHJhaW4pLCAzKQpwcmludChtaXNzX3Byb3ApIAp3aGljaChtaXNzX3Byb3A+MC41KSAjIGZvdXIgZmVhdHVyZXMgaGF2ZSBncmVhdGVyIHRoYW4gNTAlIG9mIGRhdGEgIm1pc3NpbmciIC0tIGRyb3AgdGhlc2UgdmFyaWFibGVzCmBgYAoKCk5vdGVzIGFib3V0IGRhdGEgY2xlYW5pbmc6CgpXZSBkcm9wcGVkIOKAnHV0aWxpdGllc+KAnSAodHlwZSBvZiB1dGlsaXRpZXMgYXZhaWxhYmxlKSBzaW5jZSBpbiB0aGUgdHJhaW5pbmcgc2V0LCBvbmx5IDIgb2JzZXJ2YXRpb25zIGRpZCBub3QgaGF2ZSBhbGwgdGhlIHV0aWxpdGllcyAoZWxlY3RyaWNpdHksIGdhcywgd2F0ZXIgYW5kIHNld2FnZSkuIEludHVpdGl2ZWx5LCBtb3N0IG1vZGVybiBwcm9wZXJ0eSBhcmUgZXF1aXBwZWQgd2l0aCB0aGVzZSBiYXNpYyBwdWJsaWMgdXRpbGl0aWVzIGFuZCBrZWVwaW5nIHRoZSB2YXJpYWJsZSB3b3VsZCB0aGVyZWZvcmUgYmUgdW5uZWNlc3NhcnkuIAoKV2UgYWxzbyBkcm9wcGVkIOKAnGNvbmRpdGlvbiAy4oCdIChwcm94aW1pdHkgdG8gdmFyaW91cyBjb25kaXRpb25zIGlmIG1vcmUgdGhhbiBvbmUgaXMgcHJlc2VudCkgc2luY2UgaXQgc2VlbWVkIHJlZHVuZGFudCBmcm9tIG91ciB0cmFpbmluZyBzZXQuIEdpdmVuIENvbmRpdGlvbiAxLCBvbmx5IDEyIHByb3BlcnRpZXMgd2VyZSBub3QgY2xvc2UgdG8gbm9ybWFsIGNvbmRpdGlvbnMuIAoKSW4gdGhlIG9yaWdpbmFsIHNjYWxlLCAxOTkwIGFuZCAxOTAwIHdvdWxkIG5vdCBiZSBtdWNoIGRpZmZlcmVudC4gVGhlcmVmb3JlLCB3ZSBjaGFuZ2VkIHRoZSBzY2FsZSB0byB0aGUgbnVtYmVyIG9mIHllYXJzIHNpbmNlIGxhc3QgY29uc3RydWN0aW9uIG9yIHJlbW9kZWxsaW5nLCBzdWJ0cmFjdGVkIGZyb20geWVhciAyMDEwICh0aGUgZW5kIHllYXIgaW4gdGhlIGRhdGFzZXQpLiAKCkFub3RoZXIgdmFyaWFibGUgZHJvcHBlZCBpbiBvdXIgbW9kZWwgd2FzIOKAnHJvb2YgbWF0ZXJpYWzigJ0gc2luY2Ugb25seSAxJSBvZiB0aGUgcHJvcGVydHkgdXNlZCBtYXRlcmlhbCBvdGhlciB0aGFuIOKAnHN0YW5kYXJkIGNvbXBvc2l0ZSBzaGluZ2xl4oCdLiBTaW1pbGFybHksIOKAnGhlYXRpbmfigJ0gd2FzIGFsc28gZHJvcHBlZCBzaW5jZSBtb3JlIHRoYW4gOTUlIG9mIHRoZSBwcm9wZXJ0eSBoYXMgZ2FzIGZvcmNlZCB3YXJtIGFpciBmdXJuYWNlIChHYXNBKSBpbnN0ZWFkIG9mIG90aGVyIHR5cGVzIG9mIGhlYXRpbmcuCgpGb3Ig4oCcZXh0ZXJpb3IgcXVhbGl0eeKAnSBhbmQg4oCcZXh0ZXJpb3IgY29uZGl0aW9u4oCdCSwgd2UgcmVjb2RlZCB0aGVzZSBvcmRpbmFsIHZhcmlhYmxlcyB0byAxLTUgdG8gcmVwbGFjZSB0aGUgb3JpZ2luYWwgc2NhbGUgb2YgY29uZGl0aW9ucyAoZnJvbSBwb29yIHRvIGV4Y2VsbGVudCkuIFNpbWlsYXJseSwgd2UgcmVjb2RlZCDigJxiYXNlbWVudCBleHBvc3VyZeKAnSBhbmQg4oCcYmFzZW1lbnQgcmF0aW5n4oCdIGV4Y2VwdCB0aGF0IHRoZSBuZXcgc2NhbGUgd291bGQgc3RhcnQgZnJvbSAwIGZvciBwcm9wZXJ0aWVzIHdpdGhvdXQgYmFzZW1lbnQuIAoJCQkJCQkJCQkJCQkKRXh0ZXIgUXVhbCAoT3JkaW5hbCk6IEV2YWx1YXRlcyB0aGUgcXVhbGl0eSBvZiB0aGUgbWF0ZXJpYWwgb24gdGhlIGV4dGVyaW9yCgkJCQkJCQogICAgICBFeCAgICAgICBFeGNlbGxlbnQKICAgICAgIEdkICAgICAgIEdvb2QKICAgICAgIFRBICAgICAgIEF2ZXJhZ2UvVHlwaWNhbAogICAgICAgRmEgICAgICAgRmFpcgogICAgICAgUG8gICAgICAgUG9vcgoKCgkJCQkJCQpFeHRlciBDb25kIChPcmRpbmFsKTogRXZhbHVhdGVzIHRoZSBwcmVzZW50IGNvbmRpdGlvbiBvZiB0aGUgbWF0ZXJpYWwgb24gdGhlIGV4dGVyaW9yCgogICAgICBFeCAgICAgICBFeGNlbGxlbnQKICAgICAgIEdkICAgICAgIEdvb2QKICAgICAgIFRBICAgICAgIEF2ZXJhZ2UvVHlwaWNhbAogICAgICAgRmEgICAgICAgRmFpcgogICAgICAgUG8gICAgICAgUG9vcgoKCkNvbnRpbnVvdXMgdmFyaWFibGVzIHN1Y2ggYXMg4oCcMXN0IGZsb29yIHNxdWFyZSBmZWV04oCdIGFuZCDigJwybmQgZmxvb3Igc3F1YXJlIGZlZXTigJ0gd2VyZSBsb2ctdHJhbnNmb3JtZWQgZm9yIGludGVycHJldGF0aW9uIHB1cnBvc2UuIAoKRm9yIHZhcmlhYmxlIOKAnGZ1bmN0aW9uYWzigJ0sIHdlIHJlY29kZWQgZGlmZmVyZW50IG9yZGluYWwgbGV2ZWxzIGludG8gYmluYXJ5IGxldmVscyAtLS0gdHlwaWNhbCBmdW5jdGlvbmFsaXR5IG9yIG5vdCwgaW5jbHVkaW5nIG1pbm9yIGFuZCBtYWpvciBkZWR1Y3Rpb25zLiAgCQkKCldlIHN1bW1lZCB1cCB0aGUgbnVtYmVyIG9mIGJhdGhyb29tcyB0byBvbmUgY29udGludW91cyB2YXJpYWJsZS4gTm90ZSB0aGF0IG9uZSBoYWxmLWJhdGhyb29tIHdvdWxkIGJlIGNvZGVkIGFzIDAuNS4JCmBgYHtyfQojIERpZCBub3QgcmVtb3ZlIGFueSBOQSBlbnRyaWVzIGluIExvdC5mcm9udGFnZQoKZGF0YT1hbWVzX3RyYWluCgpkYXRhIDwtIGRhdGEgICU+JQogICNmaWx0ZXIoIWlzLm5hKExvdC5Gcm9udGFnZSkpICU+JSAKICBtdXRhdGUoTVMuU3ViQ2xhc3M9IGZhY3RvcihNUy5TdWJDbGFzcykpICU+JQogIG11dGF0ZShBbGxleSA9ICBmYWN0b3IoQWxsZXksIGxldmVscyA9IGxldmVscyhhZGROQShBbGxleSkpLCBsYWJlbHMgPSBjKGxldmVscyhBbGxleSksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKEhvdXNlQWdlID0gWXIuU29sZC0gcG1heChZZWFyLkJ1aWx0LCBZZWFyLlJlbW9kLkFkZCkpICAlPiUKICBmaWx0ZXIoIWlzLm5hKE1hcy5WbnIuQXJlYSkpICAlPiUgCiAgbXV0YXRlKEJzbXQuWU4gPSAxKighaXMubmEoQnNtdC5RdWFsKSkpICU+JQogIG11dGF0ZShCc210LlF1YWwgPSAgZmFjdG9yKEJzbXQuUXVhbCwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEJzbXQuUXVhbCkpLCBsYWJlbHMgPSBjKGxldmVscyhCc210LlF1YWwpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShCc210LlF1YWwgPSByZWxldmVsKEJzbXQuUXVhbCwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShCc210LkNvbmQgPSAgZmFjdG9yKEJzbXQuQ29uZCwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEJzbXQuQ29uZCkpLCBsYWJlbHMgPSBjKGxldmVscyhCc210LkNvbmQpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShCc210LkNvbmQgPSByZWxldmVsKEJzbXQuQ29uZCwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShCc210LkV4cG9zdXJlID0gIGZhY3RvcihCc210LkV4cG9zdXJlLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoQnNtdC5FeHBvc3VyZSkpLCBsYWJlbHMgPSBjKGxldmVscyhCc210LkV4cG9zdXJlKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoQnNtdC5FeHBvc3VyZSA9IHJlbGV2ZWwoQnNtdC5FeHBvc3VyZSwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShCc210RmluLlR5cGUuMT0gIGZhY3RvcihCc210RmluLlR5cGUuMSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEJzbXRGaW4uVHlwZS4xKSksIGxhYmVscyA9IGMobGV2ZWxzKEJzbXRGaW4uVHlwZS4xKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoQnNtdEZpbi5UeXBlLjEgPSByZWxldmVsKEJzbXRGaW4uVHlwZS4xLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKEJzbXRGaW4uVHlwZS4yPSAgZmFjdG9yKEJzbXRGaW4uVHlwZS4yLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoQnNtdEZpbi5UeXBlLjIpKSwgbGFiZWxzID0gYyhsZXZlbHMoQnNtdEZpbi5UeXBlLjIpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShCc210RmluLlR5cGUuMiA9IHJlbGV2ZWwoQnNtdEZpbi5UeXBlLjIsIHJlZj0iTm9uZSIpKSAlPiUKICBtdXRhdGUoWDEyLlNGPSBYMXN0LkZsci5TRisgWDJuZC5GbHIuU0YpICAlPiUgICAKICBmaWx0ZXIoIWlzLm5hKEJzbXQuRnVsbC5CYXRoKSkgJT4lCiAgZmlsdGVyKCFpcy5uYShCc210LkhhbGYuQmF0aCkpICU+JQogIG11dGF0ZShCYXRocyA9IEJzbXQuRnVsbC5CYXRoICsgMC41KkJzbXQuSGFsZi5CYXRoICsgRnVsbC5CYXRoICsgMC41KkhhbGYuQmF0aCkgJT4lCiAgbXV0YXRlKEZpcmVwbGFjZS5ZTiA9IDEqKEZpcmVwbGFjZXM+MCkpICU+JQogIG11dGF0ZShGaXJlcGxhY2UuUXUgPSAgZmFjdG9yKEZpcmVwbGFjZS5RdSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEZpcmVwbGFjZS5RdSkpLCBsYWJlbHMgPSBjKGxldmVscyhGaXJlcGxhY2UuUXUpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShGaXJlcGxhY2UuUXUgPSByZWxldmVsKEZpcmVwbGFjZS5RdSwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShHYXJhZ2UuWU4gPSAxKighaXMubmEoR2FyYWdlLkNvbmQpKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5UeXBlID0gIGZhY3RvcihHYXJhZ2UuVHlwZSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEdhcmFnZS5UeXBlKSksIGxhYmVscyA9IGMobGV2ZWxzKEdhcmFnZS5UeXBlKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoR2FyYWdlLlR5cGUgPSByZWxldmVsKEdhcmFnZS5UeXBlLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5GaW5pc2ggPSAgZmFjdG9yKEdhcmFnZS5GaW5pc2gsIGxldmVscyA9IGxldmVscyhhZGROQShHYXJhZ2UuRmluaXNoKSksIGxhYmVscyA9IGMobGV2ZWxzKEdhcmFnZS5GaW5pc2gpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShHYXJhZ2UuRmluaXNoID0gcmVsZXZlbChHYXJhZ2UuRmluaXNoLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5RdWFsID0gIGZhY3RvcihHYXJhZ2UuUXVhbCwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEdhcmFnZS5RdWFsKSksIGxhYmVscyA9IGMobGV2ZWxzKEdhcmFnZS5RdWFsKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoR2FyYWdlLlF1YWwgPSByZWxldmVsKEdhcmFnZS5RdWFsLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5Db25kID0gIGZhY3RvcihHYXJhZ2UuQ29uZCwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEdhcmFnZS5Db25kKSksIGxhYmVscyA9IGMobGV2ZWxzKEdhcmFnZS5Db25kKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoR2FyYWdlLkNvbmQgPSByZWxldmVsKEdhcmFnZS5Db25kLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKFBvcmNoLkFyZWEgPSBXb29kLkRlY2suU0YrIE9wZW4uUG9yY2guU0YrRW5jbG9zZWQuUG9yY2grWDNTc24uUG9yY2ggKyBTY3JlZW4uUG9yY2gpICU+JQogIG11dGF0ZShQb29sLllOID0gMSooUG9vbC5BcmVhPjApKSAlPiUKICBtdXRhdGUoUG9vbC5RQyA9ICBmYWN0b3IoUG9vbC5RQywgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKFBvb2wuUUMpKSwgbGFiZWxzID0gYyhsZXZlbHMoUG9vbC5RQyksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKFBvb2wuUUMgPSByZWxldmVsKFBvb2wuUUMsIHJlZj0iTm9uZSIpKSAlPiUKICBtdXRhdGUoRmVuY2UgPSAgZmFjdG9yKEZlbmNlLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoRmVuY2UpKSwgbGFiZWxzID0gYyhsZXZlbHMoRmVuY2UpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShNaXNjLkZlYXR1cmUgPSAgZmFjdG9yKE1pc2MuRmVhdHVyZSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKE1pc2MuRmVhdHVyZSkpLCBsYWJlbHMgPSBjKGxldmVscyhNaXNjLkZlYXR1cmUpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShNby5Tb2xkID0gYXMuZmFjdG9yKE1vLlNvbGQpKSAlPiUKICBtdXRhdGUoWXIuU29sZCA9IGFzLmZhY3RvcihZci5Tb2xkKSkgJT4lCiAgZHBseXI6OnNlbGVjdCgtR2FyYWdlLllyLkJsdCkgJT4lCiAgbXV0YXRlKENvbmRpdGlvbi4xID0gYXMuY2hhcmFjdGVyKENvbmRpdGlvbi4xKSkgJT4lCiAgbXV0YXRlKEtpdGNoZW4uUXVhbD1wbHlyOjptYXB2YWx1ZXMoS2l0Y2hlbi5RdWFsLCBmcm9tID0gYygiUG8iLCAiRmEiLCAiVEEiLCJHZCIsICJFeCIgKSwgdG8gPSBjKCIxIiwgIjIiLCAiMyIsICI0IiwgIjUiKSkpICU+JQogIG11dGF0ZShLaXRjaGVuLlF1YWwgPSBhcy5udW1lcmljKGFzLmNoYXJhY3RlcihLaXRjaGVuLlF1YWwpKSkgJT4lCiAgbXV0YXRlKEhlYXRpbmcuUUM9cGx5cjo6bWFwdmFsdWVzKEhlYXRpbmcuUUMsIGZyb20gPSBjKCJQbyIsICJGYSIsICJUQSIsIkdkIiwgIkV4IiApLCB0byA9IGMoIjEiLCAiMiIsICIzIiwgIjQiLCAiNSIpKSkgJT4lCiAgbXV0YXRlKEhlYXRpbmcuUUMgPSBhcy5udW1lcmljKGFzLmNoYXJhY3RlcihIZWF0aW5nLlFDKSkpICU+JQogIG11dGF0ZShCc210LlF1YWwgPSBkcm9wbGV2ZWxzKEJzbXQuUXVhbCkpICU+JQogIG11dGF0ZShGdW5jdGlvbmFsID0gZHJvcGxldmVscyhGdW5jdGlvbmFsKSkgJT4lCiAgbXV0YXRlKFJvb2YuTWF0bCA9IGRyb3BsZXZlbHMoUm9vZi5NYXRsKSkKCiMgU2ltcGxpZnkgQ29uZGl0aW9uIDEgKFBhcmssIFJhaWwsIE5vcm1hbCkKaW5kX3JhaWw8LXdoaWNoKGRhdGEkQ29uZGl0aW9uLjE9PSJSUk5uIiB8IGRhdGEkQ29uZGl0aW9uLjE9PSJSUkFuIiB8IGRhdGEkQ29uZGl0aW9uLjE9PSJSUk5lIiB8IGRhdGEkQ29uZGl0aW9uLjE9PSJSUkFlIikKaW5kX3Bhcms8LXdoaWNoKGRhdGEkQ29uZGl0aW9uLjE9PSJQb3NOIiB8IGRhdGEkQ29uZGl0aW9uLjE9PSJQb3NBIikKZGF0YSRDb25kaXRpb24uMVtpbmRfcmFpbF08LSJSYWlsIgpkYXRhJENvbmRpdGlvbi4xW2luZF9wYXJrXTwtIlBhcmsiCmRhdGEgPSBkYXRhICU+JSAKICBtdXRhdGUoQ29uZGl0aW9uLjEgPSBmYWN0b3IoQ29uZGl0aW9uLjEpKSAlPiUKICBtdXRhdGUoQ29uZGl0aW9uLjEgPSByZWxldmVsKENvbmRpdGlvbi4xLCByZWY9Ik5vcm0iKSkgCgojIEVsaW1pbmF0ZSB0aGUgb25lIGVudHJ5IGluICdFeHBvc3VyZScgdGhhdCBoYWQgYmVlbiBsZWZ0IGNvbXBsZXRlbHkgZW1wdHkKZGF0YV90cmFpbjwtZGF0YQpkYXRhX3RyYWluJEJzbXQuRXhwb3N1cmVbd2hpY2goZGF0YV90cmFpbiRCc210LkV4cG9zdXJlPT0iIildPC0iTm9uZSIKZGF0YV90cmFpbiRCc210LkV4cG9zdXJlPC1kcm9wbGV2ZWxzKGRhdGFfdHJhaW4kQnNtdC5FeHBvc3VyZSkKCmRhdGFfdHJhaW4kUG9vbC5BcmVhPC1kYXRhX3RyYWluJFBvb2wuQXJlYSsxCmRhdGFfdHJhaW4kVG90YWwuQnNtdC5TRjwtZGF0YV90cmFpbiRUb3RhbC5Cc210LlNGKzEKYGBgCgpUaGUgYE5laWdoYm9yaG9vZGAgdmFyaWFibGUsIHR5cGljYWxseSBvZiBsaXR0bGUgaW50ZXJlc3Qgb3RoZXIgdGhhbiB0byBtb2RlbCB0aGUgbG9jYXRpb24gZWZmZWN0LCBtYXkgYmUgb2YgbW9yZSByZWxldmFuY2Ugd2hlbiB1c2VkIHdpdGggdGhlIFttYXBdKGh0dHA6Ly93d3cuYW1zdGF0Lm9yZy9wdWJsaWNhdGlvbnMvanNlL3YxOW4zL2RlY29jay9BbWVzUmVzaWRlbnRpYWwucGRmKS4KCldlIGFyZSByZXN0cmljdGluZyBhdHRlbnRpb24gdG8ganVzdCB0aGUgIm5vcm1hbCBzYWxlcyIgY29uZGl0aW9uLgoKCiMjIFBhcnQgSTogU2ltcGxlIE1vZGVsCgpJbiB0aGUgZmlyc3QgbW9kZWwgeW91IGFyZSBhbGxvd2VkIG9ubHkgbGltaXRlZCBtYW5pcHVsYXRpb25zIG9mIHRoZSBvcmlnaW5hbCBkYXRhIHNldCB0byBwcmVkaWN0IHRoZSBzYWxlcyBwcmljZSBgcHJpY2VgLiBZb3UgYXJlIGFsbG93ZWQgdG8gdGFrZSBwb3dlciB0cmFuc2Zvcm1hdGlvbnMgb2YgdGhlIG9yaWdpbmFsIHZhcmlhYmxlcyBbc3F1YXJlIHJvb3RzLCBsb2dzLCBpbnZlcnNlcywgc3F1YXJlcywgZXRjLl0gYnV0IHlvdSBhcmUgTk9UIGFsbG93ZWQgdG8gY3JlYXRlIGludGVyYWN0aW9uIHZhcmlhYmxlcy4gVGhpcyBtZWFucyB0aGF0IGEgdmFyaWFibGUgbWF5IG9ubHkgYmUgdXNlZCBvbmNlIGluIGFuIGVxdWF0aW9uIFtpZiB5b3UgdXNlICQgeF4yJCBkb27igJl0IHVzZSAkeCRdLiBBZGRpdGlvbmFsbHksIHlvdSBtYXkgZWxpbWluYXRlIGFueSBkYXRhIHBvaW50cyB5b3UgZGVlbSB1bmZpdC4gVGhpcyBtb2RlbCBzaG91bGQgaGF2ZSBhIG1pbmltdW0gci1zcXVhcmUgb2YgNzMlIChpbiB0aGUgb3JpZ2luYWwgdW5pdHMpIGFuZCBjb250YWluIGF0IGxlYXN0IDYgdmFyaWFibGVzIGJ1dCBmZXdlciB0aGFuIDIwLiAgIAoKCgoKYGBge3J9CiMjIyBwZXJmcm9tYW5jZSBldmxhdXRpb24gZnVuY3Rpb24KcGVyZm9ybWFuY2U8LSBmdW5jdGlvbihZLCBZaGF0KXsKICAKICBiaWFzPC0gbWVhbihZLVloYXRbLDFdKQogIAogIG1heC5kZXY8LW1heChhYnMoWS1ZaGF0WywxXSkpCiAgCiAgbWVhbi5kZXY8LW1lYW4oYWJzKFktWWhhdFssMV0pKQogIAogIFJNU0U8LXNxcnQobWVhbigoWS1ZaGF0WywxXSleMikpCiAgCiAgY292ZXJhZ2U8LW1lYW4oKFk+WWhhdFssMl0pICYgKFk8WWhhdFssM10pKQogIAogIG91dDwtZGF0YS5mcmFtZShiaWFzPWJpYXMsIG1heC5kZXY9bWF4LmRldiwgbWVhbi5kZXY9bWVhbi5kZXYsIFJNRVM9Uk1TRSwgQ292ZXJhZ2U9Y292ZXJhZ2UpCiAgCiAgcmV0dXJuKG91dCkKfQpgYGAKCgoKCmBgYHtyfQpsaWJyYXJ5KE1BU1MpCgojIEJhc2UgbW9kZWwgd2l0aCB0cmFuc2Zvcm1lZCBwcmVkaWN0b3JzIAogbW9kZWw9bG0ocHJpY2UgfiBNUy5TdWJDbGFzcyArIE1TLlpvbmluZyArIGxvZyhMb3QuRnJvbnRhZ2UpICsgbG9nKExvdC5BcmVhKSArICBTdHJlZXQgKyBBbGxleSArIExvdC5TaGFwZSArIExhbmQuQ29udG91ciArIExvdC5Db25maWcgKyBMYW5kLlNsb3BlICsgTmVpZ2hib3Job29kICsgQ29uZGl0aW9uLjEgICsgQmxkZy5UeXBlICsgSG91c2UuU3R5bGUgKyBPdmVyYWxsLlF1YWwgKyBPdmVyYWxsLkNvbmQgKyBIb3VzZUFnZSArICAgUm9vZi5TdHlsZSArIFJvb2YuTWF0bCArIEV4dGVyaW9yLjFzdCArIE1hcy5WbnIuVHlwZSArIGxvZygxK01hcy5WbnIuQXJlYSkgKyBFeHRlci5Db25kICsgRXh0ZXIuUXVhbCArICAgRm91bmRhdGlvbiArIEJzbXQuUXVhbCArIEJzbXQuQ29uZCArIEJzbXQuRXhwb3N1cmUgKyBUb3RhbC5Cc210LlNGICsgSGVhdGluZyArIEhlYXRpbmcuUUMgKyBDZW50cmFsLkFpciArIEVsZWN0cmljYWwgKyBsb2coWDEyLlNGKSArIGxvZygxK0xvdy5RdWFsLkZpbi5TRikgKyBCYXRocyArIEJlZHJvb20uQWJ2R3IgKyBLaXRjaGVuLkFidkdyICsgS2l0Y2hlbi5RdWFsICsgRnVuY3Rpb25hbCArIEZpcmVwbGFjZXMgKyBGaXJlcGxhY2UuUXUgKyBHYXJhZ2UuVHlwZSArIEdhcmFnZS5GaW5pc2ggKyBHYXJhZ2UuQ2FycyArIEdhcmFnZS5Db25kICsgR2FyYWdlLlF1YWwgKyBQYXZlZC5Ecml2ZSArIGxvZygxK1Bvb2wuQXJlYSkgKyBQb29sLlFDICsgRmVuY2UgKyBNaXNjLlZhbCArIE1vLlNvbGQgICtZci5Tb2xkICsgU2FsZS5UeXBlICArIFRvdGFsU3EsIGRhdGE9ZGF0YV90cmFpbikKCgojIEJveGNveCAoaW5kaWNhdGVzIHRoYXQgbG9nIGlzIGRlY2VudCkKbDwtYm94Y294KG1vZGVsKQpleHBvPC1yb3VuZChsJHhbd2hpY2gubWF4KGwkeSldLDIpCgojIyBDdXJyZW50IG1vZGVsCiMjIGxvZyhMb3QuRnJvbnRhZ2UpIGN1cnJlbnRseSByZW1vdmVkIHRvIGhhdmUgbW9yZSBkYXRhIHBvaW50cyAod2FzIG5vdCBpbmNsdWRlZCB3aGVuIGxlZnQgaW4gdGhlIG1vZGVsIHdpdGggQklDKQptb2RlbC4wPWxtKGxvZyhwcmljZSkgfiBNUy5TdWJDbGFzcyArIE1TLlpvbmluZyAgKyBsb2coTG90LkFyZWEpICsgIFN0cmVldCArIEFsbGV5ICsgTG90LlNoYXBlICsgTGFuZC5Db250b3VyICsgTG90LkNvbmZpZyArIExhbmQuU2xvcGUgKyBOZWlnaGJvcmhvb2QgKyBDb25kaXRpb24uMSAgKyBCbGRnLlR5cGUgKyBIb3VzZS5TdHlsZSArIE92ZXJhbGwuUXVhbCArIE92ZXJhbGwuQ29uZCArIEhvdXNlQWdlICsgICBSb29mLlN0eWxlICsgUm9vZi5NYXRsICsgRXh0ZXJpb3IuMXN0ICsgIE1hcy5WbnIuVHlwZSArIGxvZygxK01hcy5WbnIuQXJlYSkgKyBFeHRlci5Db25kICsgRXh0ZXIuUXVhbCArICAgRm91bmRhdGlvbiArIEJzbXQuUXVhbCArIEJzbXQuQ29uZCArIEJzbXQuRXhwb3N1cmUgKyBsb2coVG90YWwuQnNtdC5TRikgKyBCc210LllOKyBIZWF0aW5nICsgSGVhdGluZy5RQyArIENlbnRyYWwuQWlyICArIGxvZygxK0xvdy5RdWFsLkZpbi5TRikgKyBCYXRocyArIEJlZHJvb20uQWJ2R3IgKyBLaXRjaGVuLkFidkdyICsgS2l0Y2hlbi5RdWFsICsgRnVuY3Rpb25hbCArIEZpcmVwbGFjZXMgKyBGaXJlcGxhY2UuUXUgKyBHYXJhZ2UuVHlwZSArIEdhcmFnZS5GaW5pc2ggKyBHYXJhZ2UuQ2FycyArR2FyYWdlLlF1YWwrR2FyYWdlLkNvbmQgICsgUGF2ZWQuRHJpdmUgKyBsb2coUG9vbC5BcmVhKSArIFBvb2wuUUMgKyBGZW5jZSArIE1pc2MuVmFsICsgTW8uU29sZCAgK1lyLlNvbGQgKyBTYWxlLlR5cGUgICsgbG9nKFRvdGFsU3EpICsgUG9vbC5ZTiAsIGRhdGE9ZGF0YV90cmFpbikKCgojIFRoZXJlIGFyZSBzb21lIHBlcmZlY3QgY29sbGluZWFyaXRpZXMgaW4gdGhpcyBtb2RlbCAtPiBlbGltaW5hdGUgaXQgdmlhIEFJQy9CSUMKc3VtbWFyeShtb2RlbC4wKQpwbG90KG1vZGVsLjApCgojQUlDCiNtb2RlbC5BSUM9c3RlcChtb2RlbC4wLCBrPTIpCiNzdW1tYXJ5KG1vZGVsLkFJQykKI3Bsb3QobW9kZWwuQUlDKQoKI0JJQwojbW9kZWwuQklDPXN0ZXAobW9kZWwuMCwgaz1sb2cobnJvdyhkYXRhX3RyYWluKSkpCgoKbW9kZWwuQklDPWxtKGZvcm11bGEgPSBsb2cocHJpY2UpIH4gbG9nKExvdC5BcmVhKSArIE5laWdoYm9yaG9vZCArIENvbmRpdGlvbi4xICsgCiAgICAgICAgICAgICAgIE92ZXJhbGwuUXVhbCArIE92ZXJhbGwuQ29uZCArIEhvdXNlQWdlICsgRm91bmRhdGlvbiArIEJzbXQuUXVhbCArIAogICAgICAgICAgICAgICBCc210LkV4cG9zdXJlICsgbG9nKFRvdGFsLkJzbXQuU0YpICsgSGVhdGluZy5RQyArIENlbnRyYWwuQWlyICsgCiAgICAgICAgICAgICAgIEJhdGhzICsgQmVkcm9vbS5BYnZHciArIEtpdGNoZW4uQWJ2R3IgKyBLaXRjaGVuLlF1YWwgKyBGdW5jdGlvbmFsICsgCiAgICAgICAgICAgICAgIEZpcmVwbGFjZXMgKyBHYXJhZ2UuQ2FycyArIFBhdmVkLkRyaXZlICsgbG9nKFBvb2wuQXJlYSkgKyAKICAgICAgICAgICAgICAgbG9nKFRvdGFsU3EpLCBkYXRhID0gZGF0YV90cmFpbikKCgpzdW1tYXJ5KG1vZGVsLkJJQykKcGxvdChtb2RlbC5CSUMpCgojIEV4cGxvcmluZyB0aGUgcmVtYWluaW5nIHByZWRpY3RvcnMgcmVsYXRpb25zaGlwIHRvIHByaWNlCiMgcGxvdChsb2coZGF0YV90cmFpbiRMb3QuQXJlYSksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKIyBwbG90KChkYXRhX3RyYWluJE5laWdoYm9yaG9vZCksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKIyBwbG90KChkYXRhX3RyYWluJENvbmRpdGlvbi4xKSwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpKQojIHBsb3QoKGRhdGFfdHJhaW4kT3ZlcmFsbC5RdWFsKSwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpKQojIHBsb3QoKGRhdGFfdHJhaW4kT3ZlcmFsbC5Db25kKSwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpKQojIHBsb3QoKGRhdGFfdHJhaW4kSG91c2VBZ2UpLCBsb2coZGF0YV90cmFpbiRwcmljZSkpCiMgcGxvdCgoZGF0YV90cmFpbiRCc210LlF1YWwpLCBsb2coZGF0YV90cmFpbiRwcmljZSkpCiMgcGxvdCgoZGF0YV90cmFpbiRCc210LkV4cG9zdXJlKSwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpKQojIHBsb3QobG9nKGRhdGFfdHJhaW4kVG90YWwuQnNtdC5TRiksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKIyBwbG90KChkYXRhX3RyYWluJEhlYXRpbmcuUUMpLCBsb2coZGF0YV90cmFpbiRwcmljZSkpCiMgcGxvdCgoZGF0YV90cmFpbiRDZW50cmFsLkFpciksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKIyBwbG90KChkYXRhX3RyYWluJEJhdGhzKSwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpKQojIHBsb3QoKGRhdGFfdHJhaW4kQmVkcm9vbS5BYnZHciksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKIyBwbG90KChkYXRhX3RyYWluJEtpdGNoZW4uQWJ2R3IpLCBsb2coZGF0YV90cmFpbiRwcmljZSkpCiMgcGxvdCgoZGF0YV90cmFpbiRLaXRjaGVuLlF1YWwpLCBsb2coZGF0YV90cmFpbiRwcmljZSkpCiMgcGxvdCgoZGF0YV90cmFpbiRGdW5jdGlvbmFsKSwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpKQojIHBsb3QoKGRhdGFfdHJhaW4kRmlyZXBsYWNlcyksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKIyBwbG90KChkYXRhX3RyYWluJFBhdmVkLkRyaXZlKSwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpKQojIHBsb3QoKGRhdGFfdHJhaW4kR2FyYWdlLkNhcnMpLCBsb2coZGF0YV90cmFpbiRwcmljZSkpCiMgcGxvdCgoZGF0YV90cmFpbiRHYXJhZ2UuQ2FycyksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKIyBwbG90KGxvZygxK2RhdGFfdHJhaW4kUG9vbC5BcmVhKSwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpKQojIHBsb3QobG9nKGRhdGFfdHJhaW4kVG90YWxTcSksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKCnRlcm1wbG90KG1vZGVsLkJJQywgcGFydGlhbC5yZXNpZCA9IFRSVUUsIGNvbC5yZXMgPSAicHVycGxlIiwgY2V4ID0gMC41LAogICAgICAgICBydWcgPSBULCBzZSA9IFQsIHNtb290aCA9IHBhbmVsLnNtb290aCkKCgojIFRoZXJlIGFyZSAzIGhpZ2ggbGV2ZXJhZ2UgcG9pbnRzIC0gbWF5IHdhbnQgdG8gZXhjbHVkZSB0aGVtCmhoPC1oYXR2YWx1ZXMobW9kZWwuQklDKQppZDwtd2hpY2goaGg9PTEpCgpwbG90KGhhdHZhbHVlcyhtb2RlbC5CSUMpLCB0eXBlID0gImgiKQpgYGAKCgoKCgoKCmBgYHtyfQojIyBQcmVwYXJlIHRlc3QgZGF0YQpsb2FkKCJhbWVzX3Rlc3QuUmRhdGEiKQoKZGF0YT1hbWVzX3Rlc3QKCmRhdGEgPC0gZGF0YSAgJT4lICAKICAjZmlsdGVyKCFpcy5uYShMb3QuRnJvbnRhZ2UpKSAlPiUgCiAgbXV0YXRlKE1TLlN1YkNsYXNzPSBmYWN0b3IoTVMuU3ViQ2xhc3MpKSAlPiUKICBtdXRhdGUoQWxsZXkgPSAgZmFjdG9yKEFsbGV5LCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoQWxsZXkpKSwgbGFiZWxzID0gYyhsZXZlbHMoQWxsZXkpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShIb3VzZUFnZSA9IFlyLlNvbGQtIHBtYXgoWWVhci5CdWlsdCwgWWVhci5SZW1vZC5BZGQpKSAgJT4lCiAgI2ZpbHRlcighaXMubmEoTWFzLlZuci5BcmVhKSkgICU+JSAKICBtdXRhdGUoQnNtdC5ZTiA9IDEqKCFpcy5uYShCc210LlF1YWwpKSkgJT4lCiAgbXV0YXRlKEJzbXQuUXVhbCA9ICBmYWN0b3IoQnNtdC5RdWFsLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoQnNtdC5RdWFsKSksIGxhYmVscyA9IGMobGV2ZWxzKEJzbXQuUXVhbCksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKEJzbXQuUXVhbCA9IHJlbGV2ZWwoQnNtdC5RdWFsLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKEJzbXQuQ29uZCA9ICBmYWN0b3IoQnNtdC5Db25kLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoQnNtdC5Db25kKSksIGxhYmVscyA9IGMobGV2ZWxzKEJzbXQuQ29uZCksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKEJzbXQuQ29uZCA9IHJlbGV2ZWwoQnNtdC5Db25kLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKEJzbXQuRXhwb3N1cmUgPSAgZmFjdG9yKEJzbXQuRXhwb3N1cmUsIGxldmVscyA9IGxldmVscyhhZGROQShCc210LkV4cG9zdXJlKSksIGxhYmVscyA9IGMobGV2ZWxzKEJzbXQuRXhwb3N1cmUpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShCc210LkV4cG9zdXJlID0gcmVsZXZlbChCc210LkV4cG9zdXJlLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKEJzbXRGaW4uVHlwZS4xPSAgZmFjdG9yKEJzbXRGaW4uVHlwZS4xLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoQnNtdEZpbi5UeXBlLjEpKSwgbGFiZWxzID0gYyhsZXZlbHMoQnNtdEZpbi5UeXBlLjEpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShCc210RmluLlR5cGUuMSA9IHJlbGV2ZWwoQnNtdEZpbi5UeXBlLjEsIHJlZj0iTm9uZSIpKSAlPiUKICBtdXRhdGUoQnNtdEZpbi5UeXBlLjI9ICBmYWN0b3IoQnNtdEZpbi5UeXBlLjIsIGxldmVscyA9IGxldmVscyhhZGROQShCc210RmluLlR5cGUuMikpLCBsYWJlbHMgPSBjKGxldmVscyhCc210RmluLlR5cGUuMiksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKEJzbXRGaW4uVHlwZS4yID0gcmVsZXZlbChCc210RmluLlR5cGUuMiwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShYMTIuU0Y9IFgxc3QuRmxyLlNGKyBYMm5kLkZsci5TRikgICU+JSAgIAogICNmaWx0ZXIoIWlzLm5hKEJzbXQuRnVsbC5CYXRoKSkgJT4lCiAgI2ZpbHRlcighaXMubmEoQnNtdC5IYWxmLkJhdGgpKSAlPiUKICBtdXRhdGUoQmF0aHMgPSBCc210LkZ1bGwuQmF0aCArIDAuNSpCc210LkhhbGYuQmF0aCArIEZ1bGwuQmF0aCArIDAuNSpIYWxmLkJhdGgpICU+JQogIG11dGF0ZShGaXJlcGxhY2UuWU4gPSAxKihGaXJlcGxhY2VzPjApKSAlPiUKICBtdXRhdGUoRmlyZXBsYWNlLlF1ID0gIGZhY3RvcihGaXJlcGxhY2UuUXUsIGxldmVscyA9IGxldmVscyhhZGROQShGaXJlcGxhY2UuUXUpKSwgbGFiZWxzID0gYyhsZXZlbHMoRmlyZXBsYWNlLlF1KSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoRmlyZXBsYWNlLlF1ID0gcmVsZXZlbChGaXJlcGxhY2UuUXUsIHJlZj0iTm9uZSIpKSAlPiUKICBtdXRhdGUoR2FyYWdlLllOID0gMSooIWlzLm5hKEdhcmFnZS5Db25kKSkpICU+JQogIG11dGF0ZShHYXJhZ2UuVHlwZSA9ICBmYWN0b3IoR2FyYWdlLlR5cGUsIGxldmVscyA9IGxldmVscyhhZGROQShHYXJhZ2UuVHlwZSkpLCBsYWJlbHMgPSBjKGxldmVscyhHYXJhZ2UuVHlwZSksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5UeXBlID0gcmVsZXZlbChHYXJhZ2UuVHlwZSwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShHYXJhZ2UuRmluaXNoID0gIGZhY3RvcihHYXJhZ2UuRmluaXNoLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoR2FyYWdlLkZpbmlzaCkpLCBsYWJlbHMgPSBjKGxldmVscyhHYXJhZ2UuRmluaXNoKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoR2FyYWdlLkZpbmlzaCA9IHJlbGV2ZWwoR2FyYWdlLkZpbmlzaCwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShHYXJhZ2UuUXVhbCA9ICBmYWN0b3IoR2FyYWdlLlF1YWwsIGxldmVscyA9IGxldmVscyhhZGROQShHYXJhZ2UuUXVhbCkpLCBsYWJlbHMgPSBjKGxldmVscyhHYXJhZ2UuUXVhbCksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5RdWFsID0gcmVsZXZlbChHYXJhZ2UuUXVhbCwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShHYXJhZ2UuQ29uZCA9ICBmYWN0b3IoR2FyYWdlLkNvbmQsIGxldmVscyA9IGxldmVscyhhZGROQShHYXJhZ2UuQ29uZCkpLCBsYWJlbHMgPSBjKGxldmVscyhHYXJhZ2UuQ29uZCksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5Db25kID0gcmVsZXZlbChHYXJhZ2UuQ29uZCwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShQb3JjaC5BcmVhID0gV29vZC5EZWNrLlNGKyBPcGVuLlBvcmNoLlNGK0VuY2xvc2VkLlBvcmNoK1gzU3NuLlBvcmNoICsgU2NyZWVuLlBvcmNoKSAlPiUKICBtdXRhdGUoUG9vbC5ZTiA9IDEqKFBvb2wuQXJlYT4wKSkgJT4lCiAgbXV0YXRlKFBvb2wuUUMgPSAgZmFjdG9yKFBvb2wuUUMsIGxldmVscyA9IGxldmVscyhhZGROQShQb29sLlFDKSksIGxhYmVscyA9IGMobGV2ZWxzKFBvb2wuUUMpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShQb29sLlFDID0gcmVsZXZlbChQb29sLlFDLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKEZlbmNlID0gIGZhY3RvcihGZW5jZSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEZlbmNlKSksIGxhYmVscyA9IGMobGV2ZWxzKEZlbmNlKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoTWlzYy5GZWF0dXJlID0gIGZhY3RvcihNaXNjLkZlYXR1cmUsIGxldmVscyA9IGxldmVscyhhZGROQShNaXNjLkZlYXR1cmUpKSwgbGFiZWxzID0gYyhsZXZlbHMoTWlzYy5GZWF0dXJlKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoTW8uU29sZCA9IGFzLmZhY3RvcihNby5Tb2xkKSkgJT4lCiAgbXV0YXRlKFlyLlNvbGQgPSBhcy5mYWN0b3IoWXIuU29sZCkpICU+JQogIGRwbHlyOjpzZWxlY3QoLUdhcmFnZS5Zci5CbHQpICU+JQogIG11dGF0ZShDb25kaXRpb24uMSA9IGFzLmNoYXJhY3RlcihDb25kaXRpb24uMSkpICU+JQogIG11dGF0ZShLaXRjaGVuLlF1YWw9cGx5cjo6bWFwdmFsdWVzKEtpdGNoZW4uUXVhbCwgZnJvbSA9IGMoIlBvIiwgIkZhIiwgIlRBIiwiR2QiLCAiRXgiICksIHRvID0gYygiMSIsICIyIiwgIjMiLCAiNCIsICI1IikpKSAlPiUKICBtdXRhdGUoS2l0Y2hlbi5RdWFsID0gYXMubnVtZXJpYyhhcy5jaGFyYWN0ZXIoS2l0Y2hlbi5RdWFsKSkpICU+JQogIG11dGF0ZShIZWF0aW5nLlFDPXBseXI6Om1hcHZhbHVlcyhIZWF0aW5nLlFDLCBmcm9tID0gYygiUG8iLCAiRmEiLCAiVEEiLCJHZCIsICJFeCIgKSwgdG8gPSBjKCIxIiwgIjIiLCAiMyIsICI0IiwgIjUiKSkpICU+JQogIG11dGF0ZShIZWF0aW5nLlFDID0gYXMubnVtZXJpYyhhcy5jaGFyYWN0ZXIoSGVhdGluZy5RQykpKSAlPiUKICBtdXRhdGUoQnNtdC5RdWFsID0gZHJvcGxldmVscyhCc210LlF1YWwpKSAlPiUKICBtdXRhdGUoRnVuY3Rpb25hbCA9IGRyb3BsZXZlbHMoRnVuY3Rpb25hbCkpICU+JQogIG11dGF0ZShSb29mLk1hdGwgPSBkcm9wbGV2ZWxzKFJvb2YuTWF0bCkpCgppbmRfcmFpbDwtd2hpY2goZGF0YSRDb25kaXRpb24uMT09IlJSTm4iIHwgZGF0YSRDb25kaXRpb24uMT09IlJSQW4iIHwgZGF0YSRDb25kaXRpb24uMT09IlJSTmUiIHwgZGF0YSRDb25kaXRpb24uMT09IlJSQWUiKQppbmRfcGFyazwtd2hpY2goZGF0YSRDb25kaXRpb24uMT09IlBvc04iIHwgZGF0YSRDb25kaXRpb24uMT09IlBvc0EiKQpkYXRhJENvbmRpdGlvbi4xW2luZF9yYWlsXTwtIlJhaWwiCmRhdGEkQ29uZGl0aW9uLjFbaW5kX3BhcmtdPC0iUGFyayIKCmRhdGEgPSBkYXRhICU+JSAKICBtdXRhdGUoQ29uZGl0aW9uLjEgPSBmYWN0b3IoQ29uZGl0aW9uLjEpKSAlPiUKICBtdXRhdGUoQ29uZGl0aW9uLjEgPSByZWxldmVsKENvbmRpdGlvbi4xLCByZWY9Ik5vcm0iKSkgCgpkYXRhX3Rlc3Q9ZGF0YQoKCmRhdGFfdGVzdCRCc210LkV4cG9zdXJlW3doaWNoKGRhdGFfdGVzdCRCc210LkV4cG9zdXJlPT0iIildPC0iTm9uZSIKZGF0YV90ZXN0JEJzbXQuRXhwb3N1cmU8LWRyb3BsZXZlbHMoZGF0YV90ZXN0JEJzbXQuRXhwb3N1cmUpCgoKZGF0YV90ZXN0JFBvb2wuQXJlYTwtZGF0YV90ZXN0JFBvb2wuQXJlYSsxCmRhdGFfdGVzdCRUb3RhbC5Cc210LlNGPC1kYXRhX3Rlc3QkVG90YWwuQnNtdC5TRisxCgoKIyBleHRyYWN0IHRoZSB0cnV0aApZID0gZGF0YV90ZXN0JHByaWNlCgojIG1ha2UgcHJlZGljdGlvbiBiYXNlZCBvbiBzcGVjaWZpYyBtb2RlbApZaGF0ID0gcHJlZGljdChtb2RlbC5CSUMsIG5ld2RhdGE9ZGF0YV90ZXN0LCBpbnRlcnZhbD0icHJlZGljdCIpCgojIGRlcGVuZGluZyBvbiB0aGUgcmVzcG9uc2UgdHJhbnNmb3JtYXRpb24KWWhhdCA9IGV4cChZaGF0KQoKCiMgbmFtZSBkYXRhZnJhbWUgYXMgcHJlZGljdGlvbnMhIERPIE5PVCBDSEFOR0UKcHJlZGljdGlvbnMgPSBhcy5kYXRhLmZyYW1lKFloYXQpCnByZWRpY3Rpb25zJFBJRCA9IGRhdGFfdGVzdCRQSUQKCnNhdmUocHJlZGljdGlvbnMsIGZpbGU9InByZWRpY3QuUmRhdGEiKQoKcGVyZm9ybWFuY2UoWSwgWWhhdCkKYGBgCgoKYGBge3J9CiMgTGFzc28gZm9yIHRoZSBzaW1wbGlmaWVkIG1vZGVsIChmcm9tIEJJQykKbGlicmFyeShnbG1uZXQpCgpYLnRyYWluID0gbW9kZWwubWF0cml4KGxvZyhwcmljZSkgfiBsb2coTG90LkFyZWEpICsgTmVpZ2hib3Job29kICsgQ29uZGl0aW9uLjEgKyAKICAgICAgICAgICAgICAgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgSG91c2VBZ2UgKyBGb3VuZGF0aW9uICsgQnNtdC5RdWFsICsgCiAgICAgICAgICAgICAgIEJzbXQuRXhwb3N1cmUgKyBsb2coVG90YWwuQnNtdC5TRikgKyBIZWF0aW5nLlFDICsgQ2VudHJhbC5BaXIgKyAKICAgICAgICAgICAgICAgQmF0aHMgKyBCZWRyb29tLkFidkdyICsgS2l0Y2hlbi5BYnZHciArIEtpdGNoZW4uUXVhbCArIEZ1bmN0aW9uYWwgKyAKICAgICAgICAgICAgICAgRmlyZXBsYWNlcyArIEdhcmFnZS5DYXJzICsgUGF2ZWQuRHJpdmUgKyBsb2coUG9vbC5BcmVhKSArIAogICAgICAgICAgICAgICBsb2coVG90YWxTcSksIGRhdGEgPSBkYXRhX3RyYWluKQoKClgudGVzdCA9IG1vZGVsLm1hdHJpeChsb2cocHJpY2UpIH4gbG9nKExvdC5BcmVhKSArIE5laWdoYm9yaG9vZCArIENvbmRpdGlvbi4xICsgCiAgICAgICAgICAgICAgIE92ZXJhbGwuUXVhbCArIE92ZXJhbGwuQ29uZCArIEhvdXNlQWdlICsgRm91bmRhdGlvbiArIEJzbXQuUXVhbCArIAogICAgICAgICAgICAgICBCc210LkV4cG9zdXJlICsgbG9nKFRvdGFsLkJzbXQuU0YpICsgSGVhdGluZy5RQyArIENlbnRyYWwuQWlyICsgCiAgICAgICAgICAgICAgIEJhdGhzICsgQmVkcm9vbS5BYnZHciArIEtpdGNoZW4uQWJ2R3IgKyBLaXRjaGVuLlF1YWwgKyBGdW5jdGlvbmFsICsgCiAgICAgICAgICAgICAgIEZpcmVwbGFjZXMgKyBHYXJhZ2UuQ2FycyArIFBhdmVkLkRyaXZlICsgbG9nKFBvb2wuQXJlYSkgKyAKICAgICAgICAgICAgICAgbG9nKFRvdGFsU3EpLCBkYXRhID0gZGF0YV90ZXN0KQoKCm1vZGVsLmxhc3NvID0gZ2xtbmV0KFgudHJhaW4sIGxvZyhkYXRhX3RyYWluJHByaWNlKSwgYWxwaGE9MSkKY3YubGFzc28gPSBjdi5nbG1uZXQoWC50cmFpbiwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpLCBhbHBoYT0xKQp5aGF0Lmxhc3NvID0gZXhwKHByZWRpY3QobW9kZWwubGFzc28sIHM9Y3YubGFzc28kbGFtYmRhLm1pbiwgdHlwZT0icmVzcG9uc2UiLCBuZXd4ID0gWC50ZXN0KSkKCnNxcnQobWVhbigoKHloYXQubGFzc28pLWRhdGFfdGVzdCRwcmljZSleMikpCgoKIyBSaWRnZSBmb3IgdGhlIHNpbXBsaWZpZWQgbW9kZWwgKGZyb20gQklDKQoKbW9kZWwucmlkZ2UgPSBnbG1uZXQoWC50cmFpbiwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpLCBhbHBoYT0wKQpjdi5yaWRnZSA9IGN2LmdsbW5ldChYLnRyYWluLCBsb2coZGF0YV90cmFpbiRwcmljZSksIGFscGhhPTApCnloYXQucmlkZ2UgPSBwcmVkaWN0KG1vZGVsLnJpZGdlLCBzPWN2LnJpZGdlJGxhbWJkYS5taW4sIHR5cGU9InJlc3BvbnNlIiwgbmV3eCA9IFgudGVzdCkKCnNxcnQobWVhbigoZXhwKHloYXQucmlkZ2UpLWRhdGFfdGVzdCRwcmljZSleMikpCmBgYAoKCgoKYGBge3J9CiMgQk1BIGZvciBzaW1wbGUgbW9kZWwgKGZyb20gQklDKQpsaWJyYXJ5KEJBUykKbW9kZWwuYm1hID0gYmFzLmxtKGxvZyhwcmljZSkgfiBsb2coTG90LkFyZWEpICsgTmVpZ2hib3Job29kICsgQ29uZGl0aW9uLjEgKyAKICAgICAgICAgICAgICAgICAgICAgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgSG91c2VBZ2UgKyBGb3VuZGF0aW9uICsgQnNtdC5RdWFsICsgCiAgICAgICAgICAgICAgICAgICAgIEJzbXQuRXhwb3N1cmUgKyBsb2coVG90YWwuQnNtdC5TRikgKyBIZWF0aW5nLlFDICsgQ2VudHJhbC5BaXIgKyAKICAgICAgICAgICAgICAgICAgICAgQmF0aHMgKyBCZWRyb29tLkFidkdyICsgS2l0Y2hlbi5BYnZHciArIEtpdGNoZW4uUXVhbCArIEZ1bmN0aW9uYWwgKyAKICAgICAgICAgICAgICAgICAgICAgRmlyZXBsYWNlcyArIEdhcmFnZS5DYXJzICsgUGF2ZWQuRHJpdmUgKyBsb2coUG9vbC5BcmVhKSArIAogICAgICAgICAgICAgICAgICAgICBsb2coVG90YWxTcSksIGRhdGE9ZGF0YV90cmFpbiwgIHByaW9yPSJnLXByaW9yIiwgYT1ucm93KGRhdGFfdHJhaW4pLCBtb2RlbHByaW9yPXVuaWZvcm0oKSwgbWV0aG9kPSJNQ01DIiwgTUNNQy5pdGVyYXRpb25zID0gMjAwMDAwLCB0aGluID0gMjApCgp5X3ByZWQ8LXByZWRpY3QuYmFzKG1vZGVsLmJtYSxuZXdkYXRhPWRhdGFfdGVzdCwgZXN0aW1hdG9yPSJCUE0iLCBwcmVkaWN0aW9uPVRSVUUsIHNlLmZpdD1UUlVFKQoKeV9oYXQ8LWV4cChjb25maW50LnByZWQuYmFzKHlfcHJlZCkpCgp5X2hhdD0geV9oYXRbLGMoMywxLDIpXQoKcGVyZm9ybWFuY2UoZGF0YV90ZXN0JHByaWNlLCB5X2hhdCkKYGBgCgoKCgpgYGB7cn0KbGlicmFyeShnZ3Bsb3QyKQojRGF0YSBWaXp1YWxpemF0aW9ucwojIG5laWdoYm9yaG9vZC5wcmljZS5yYW5nZSA9IGdncGxvdChkYXRhX3RyYWluLCBhZXMoeCA9IGxvZyhwcmljZSksIHkgPSByZW9yZGVyKE5laWdoYm9yaG9vZCwgZGVzYyhsb2cocHJpY2UpKSwgbWVkaWFuKSkpICsKIyAgIGdlb21fcG9seWdvbihjb2xvcj0iQmxhY2siLHNpemUgPSAxLCBhbHBoYT0xKSArIGdlb21fcG9pbnQoc3RhdD0iaWRlbnRpdHkiLCBhZXMoY29sb3I9InJlZDMiKSwgc2l6ZT0xLjUpICsgdGhlbWVfbGlnaHQoKSArIGNvb3JkX2ZsaXAoKSArIHRoZW1lKGF4aXMudGV4dC54PWVsZW1lbnRfdGV4dChhbmdsZT05MCxoanVzdD0xLHZqdXN0PTAuNSkpICsgbGFicyh5PSJOZWlnaGJvcmhvb2QiKQoKbmVpZ2hib3Job29kLnByaWNlLnJhbmdlID0gZ2dwbG90KGRhdGFfdHJhaW4sIGFlcyh5ID0gbG9nKHByaWNlKSwgeCA9IHJlb3JkZXIoTmVpZ2hib3Job29kLCBkZXNjKGxvZyhwcmljZSkpLCBtZWRpYW4pKSkgKyAKICAgICAgICAgICAgICAgICAgICAgICAgICAgZ2VvbV92aW9saW4oKSArIHRoZW1lKGF4aXMudGV4dC54PWVsZW1lbnRfdGV4dChhbmdsZT05MCxoanVzdD0xLHZqdXN0PTAuNSkpCnBsb3QobmVpZ2hib3Job29kLnByaWNlLnJhbmdlKQpgYGAKCgoKYGBge3J9CmxpYnJhcnkodHJlZW1hcCkKCnRyZWUuZGF0YSA9IGRhdGFfdHJhaW4gJT4lIG11dGF0ZShwcmljZS5iaW4gPSBpZmVsc2UocHJpY2UgPD0gNTAwMDAsICI8PSAkNTAsMDAwIiwgaWZlbHNlKHByaWNlID4gNTAwMDAgJiBwcmljZSA8PSAxMDAwMDAsICIkNTAsMDAwLTEwMCwwMDAiLCBpZmVsc2UocHJpY2UgPiAxMDAwMDAgJiBwcmljZSA8PSAyMDAwMDAsICIkMTAwLDAwMC0yMDAsMDAwIiwgaWZlbHNlKHByaWNlID4gMjAwMDAwICYgcHJpY2UgPD0gMzAwMDAwLCAiJDIwMCwwMDAtMzAwLDAwMCIsIGlmZWxzZShwcmljZSA+IDMwMDAwMCAmIHByaWNlIDw9IDQwMDAwMCwgIiQzMDAsMDAwLTQwMCwwMDAiLCBpZmVsc2UocHJpY2UgPiA0MDAwMDAgJiBwcmljZSA8PSA1MDAwMDAsICIkNDAwLDAwMC01MDAsMDAwIiwgIj4gJDUwMCwwMDAiKSkpKSkpKQoKYXJlYS5wbG90ID0gdHJlZW1hcCh0cmVlLmRhdGEsIGluZGV4PSJwcmljZS5iaW4iLCB2U2l6ZT0iTG90LkFyZWEiLCB0eXBlPSJpbmRleCIsIHBhbGV0dGU9IllsR24iLCBhbGdvcml0aG09InBpdm90U2l6ZSIsIHRpdGxlPSJFZmZlY3Qgb2YgTG90IEFyZWEgb24gUHJpY2UgUmFuZ2UiLCBmb250c2l6ZS50aXRsZSA9IDE0KQoKZGF0YV90cmFpbiRPdmVyYWxsLlF1YWwgPSBhcy5udW1lcmljKGRhdGFfdHJhaW4kT3ZlcmFsbC5RdWFsKQpxdWFsaXR5LnBsb3QgPSBnZ3Bsb3QodHJhaW4sIGFlcyh4ID0gT3ZlcmFsbC5RdWFsLCB5PXByaWNlKSkgKyBnZW9tX3Ntb290aChjb2xvcj0iZ29sZDMiKSArIGdlb21fY291bnQoY29sb3I9InJlZDMiKSArIHRoZW1lX21pbmltYWwoKQpwbG90KHF1YWxpdHkucGxvdCkKYGBgCgoKCgoKCgoKCgoKCmBgYHtyfQpsaWJyYXJ5KE1BU1MpCgojRnVsbCBNb2RlbAptb2RlbD1sbShsb2cocHJpY2UpIH4gTVMuU3ViQ2xhc3MgKyBNUy5ab25pbmcgKyBsb2coTG90LkZyb250YWdlKSArIGxvZyhMb3QuQXJlYSkgKyAgU3RyZWV0ICsgQWxsZXkgKyBMb3QuU2hhcGUgKyBMYW5kLkNvbnRvdXIgKyBMb3QuQ29uZmlnICsgTGFuZC5TbG9wZSArIE5laWdoYm9yaG9vZCArIENvbmRpdGlvbi4xICArIEJsZGcuVHlwZSArIEhvdXNlLlN0eWxlICsgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgSG91c2VBZ2UgKyBSb29mLlN0eWxlICsgUm9vZi5NYXRsICsgRXh0ZXJpb3IuMXN0ICsgRXh0ZXJpb3IuMm5kICsgTWFzLlZuci5UeXBlICsgbG9nKDErTWFzLlZuci5BcmVhKSArIEV4dGVyLlF1YWwgKyBFeHRlci5Db25kICsgRm91bmRhdGlvbiArIEJzbXQuWU46QnNtdC5RdWFsICsgQnNtdC5ZTjpCc210LkNvbmQgKyBCc210LllOOkJzbXQuRXhwb3N1cmUgKyBUb3RhbC5Cc210LlNGICsgSGVhdGluZyArIEhlYXRpbmcuUUMgKyBDZW50cmFsLkFpciArIEVsZWN0cmljYWwgKyBsb2coWDEyLlNGKSArIGxvZygxK0xvdy5RdWFsLkZpbi5TRikgKyBCYXRocyArIEJlZHJvb20uQWJ2R3IgKyBLaXRjaGVuLkFidkdyICsgS2l0Y2hlbi5RdWFsICsgRnVuY3Rpb25hbCArIEZpcmVwbGFjZXMgKyBGaXJlcGxhY2UuWU46RmlyZXBsYWNlLlF1ICsgR2FyYWdlLllOOkdhcmFnZS5UeXBlICsgR2FyYWdlLllOOkdhcmFnZS5GaW5pc2ggKyBHYXJhZ2UuQ2FycyArIEdhcmFnZS5ZTjpHYXJhZ2UuQ29uZCArIEdhcmFnZS5ZTjpHYXJhZ2UuUXVhbCArIFBhdmVkLkRyaXZlICsgbG9nKDErUG9vbC5BcmVhKSArIFBvb2wuWU46UG9vbC5RQyArIEZlbmNlICsgTWlzYy5WYWwgKyBNby5Tb2xkICsgWXIuU29sZCArIFNhbGUuVHlwZSAgKyBUb3RhbFNxLCBkYXRhPXRyYWluKQoKc3VtbWFyeShtb2RlbCkKYm94Y294KG1vZGVsKQojQklDIG1vZGVsIHNlbGVjdGlvbgpzdGVwKG1vZGVsLCBrPWxvZyhucm93KHRyYWluKSkpCgojc3RlcChtb2RlbCwgaz0yKQojIE9MRAojIG1vZGVsPWxtKGxvZyhwcmljZSl+bXMuc3ViY2xhc3MgKyBNUy5ab25pbmcgKyBsb2coTG90LkZyb250YWdlKSArIGxvZyhhcmVhKSArIFN0cmVldCArIEFsbGV5ICsgTG90LlNoYXBlICsgTGFuZC5Db250b3VyICsgTG90LkNvbmZpZyArIExhbmQuU2xvcGUgKyBOZWlnaGJvcmhvb2QgKyBDb25kaXRpb24uMSArIEJsZGcuVHlwZSArIEhvdXNlLlN0eWxlICsgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgWWVhci5CdWlsdCArIFllYXIuUmVtb2QuQWRkICsgUm9vZi5TdHlsZSArIEV4dGVyaW9yLjFzdCArIEV4dGVyaW9yLjJuZCArIE1hcy5WbnIuVHlwZSArIGxvZyhNYXMuVm5yLkFyZWEgKyAxKSArIEV4dGVyLlF1YWwgKyBFeHRlci5Db25kICsgRm91bmRhdGlvbiArIEJzbXQuUXVhbCArIEJzbXQuQ29uZCArIEJzbXQuRXhwb3N1cmUgKyBIZWF0aW5nLlFDICsgQ2VudHJhbC5BaXIgKyBFbGVjdHJpY2FsICsgbG9nKFgxc3QuRmxyLlNGKSArIGxvZyhYMm5kLkZsci5TRikgKyBCYXRocyArIEJlZHJvb20uQWJ2R3IgKyBLaXRjaGVuLkFidkdyICsgVG90Um1zLkFidkdyZCArIFBhdmVkLkRyaXZlICsgbG9nKFdvb2QuRGVjay5TRiArIDEpICsgbG9nKE9wZW4uUG9yY2guU0YgKyAxKSArIGxvZyhFbmNsb3NlZC5Qb3JjaCArIDEpICsgbG9nKFgzU3NuLlBvcmNoICsgMSkgKyBsb2coU2NyZWVuLlBvcmNoICsgMSkgKyBsb2coTWlzYy5WYWwgKyAxKSArIE1vLlNvbGQgKyBZci5Tb2xkICsgU2FsZS5UeXBlICsgbG9nKFRvdGFsU3EpLCBkYXRhPXRyYWluKQojdmlmKG1vZGVsKQojc3VtbWFyeShtb2RlbCkKCiNNb2RlbCBTZWxlY3Rpb24gYnkgQUlDCiNzdGVwKG1vZGVsLCBrPTIpCiMjQUlDX21vZGVsID0gbG0oZm9ybXVsYSA9IGxvZyhwcmljZSkgfiBQSUQgKyBhcmVhICsgTVMuU3ViQ2xhc3MgKyBNUy5ab25pbmcgKyAKIyAgICBMb3QuQXJlYSArIExvdC5TaGFwZSArIFV0aWxpdGllcyArIExhbmQuU2xvcGUgKyBOZWlnaGJvcmhvb2QgKyAKIyAgICBDb25kaXRpb24uMSArIEJsZGcuVHlwZSArIEhvdXNlLlN0eWxlICsgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgCiMgICAgWWVhci5CdWlsdCArIFllYXIuUmVtb2QuQWRkICsgUm9vZi5TdHlsZSArIFJvb2YuTWF0bCArIEV4dGVyaW9yLjFzdCArIAojICAgIEV4dGVyaW9yLjJuZCArIEV4dGVyLlF1YWwgKyBFeHRlci5Db25kICsgRm91bmRhdGlvbiArIEJzbXRGaW4uU0YuMSArIAojICAgIEJzbXRGaW4uU0YuMiArIEJzbXQuVW5mLlNGICsgSGVhdGluZyArIEhlYXRpbmcuUUMgKyBDZW50cmFsLkFpciArIAojICAgIEVsZWN0cmljYWwgKyBYMXN0LkZsci5TRiArIFgybmQuRmxyLlNGICsgQnNtdC5GdWxsLkJhdGggKyAKIyAgICBLaXRjaGVuLlF1YWwgKyBGdW5jdGlvbmFsICsgR2FyYWdlLkNhcnMgKyBHYXJhZ2UuQXJlYSArIFBhdmVkLkRyaXZlICsgCiMgICAgV29vZC5EZWNrLlNGICsgT3Blbi5Qb3JjaC5TRiArIEVuY2xvc2VkLlBvcmNoICsgU2NyZWVuLlBvcmNoICsgCiMgICAgTW8uU29sZCArIFlyLlNvbGQgKyBoYXMuZmVuY2UgKyBoYXMuZmlyZXBsYWNlLCBkYXRhID0gdHJhaW4pCiNzdW1tYXJ5KEFJQ19tb2RlbCkKCiNNb2RlbCBTZWxlY3Rpb24gYnkgQklDCiNCSUNfbW9kZWwgPSBsbShmb3JtdWxhID0gbG9nKHByaWNlKSB+IFBJRCArIGFyZWEgKyBNUy5ab25pbmcgKyBMb3QuQXJlYSArIAojICAgIE5laWdoYm9yaG9vZCArIENvbmRpdGlvbi4xICsgQmxkZy5UeXBlICsgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgCiMgICBZZWFyLkJ1aWx0ICsgWWVhci5SZW1vZC5BZGQgKyBCc210RmluLlNGLjEgKyBCc210RmluLlNGLjIgKyAKIyAgICBCc210LlVuZi5TRiArIENlbnRyYWwuQWlyICsgQnNtdC5GdWxsLkJhdGggKyBLaXRjaGVuLlF1YWwgKyAKIyAgICBGdW5jdGlvbmFsICsgR2FyYWdlLkNhcnMgKyBHYXJhZ2UuQXJlYSArIE9wZW4uUG9yY2guU0YgKyAKIyAgICBFbmNsb3NlZC5Qb3JjaCArIFNjcmVlbi5Qb3JjaCArIGhhcy5maXJlcGxhY2UsIGRhdGEgPSB0cmFpbikKI3N1bW1hcnkoQklDX21vZGVsKQoKI1VzZSBCb29zdGluZyB0byBmaW5kIGltcG9ydGFudCB2YXJpYWJsZXMKIwojbGlicmFyeShnYm0pCiMKI2Jvb3N0ID0gZ2JtKGxvZyhwcmljZSkgfiBQSUQgKyBhcmVhICsgTVMuWm9uaW5nICsgTG90LkFyZWEgKyAKIyAgICBOZWlnaGJvcmhvb2QgKyBDb25kaXRpb24uMSArIEJsZGcuVHlwZSArIE92ZXJhbGwuUXVhbCArIE92ZXJhbGwuQ29uZCArIAojICAgIFllYXIuQnVpbHQgKyBZZWFyLlJlbW9kLkFkZCArIEJzbXRGaW4uU0YuMSArIEJzbXRGaW4uU0YuMiArIAojICAgIEJzbXQuVW5mLlNGICsgQ2VudHJhbC5BaXIgKyBCc210LkZ1bGwuQmF0aCArIEtpdGNoZW4uUXVhbCArIAojICAgIEZ1bmN0aW9uYWwgKyBHYXJhZ2UuQ2FycyArIEdhcmFnZS5BcmVhICsgT3Blbi5Qb3JjaC5TRiArIAojICAgIEVuY2xvc2VkLlBvcmNoICsgU2NyZWVuLlBvcmNoICsgaGFzLmZpcmVwbGFjZSwgZGF0YSA9IHRyYWluLCBkaXN0cmlidXRpb249ImdhdXNzaWFuIiwgbi50cmVlcz01MDAwLCAjaW50ZXJhY3Rpb24uZGVwdGggPSAxLCBzaHJpbmthZ2U9MC4wMSwgdmVyYm9zZSA9IEYpCgojc3VtbWFyeShib29zdCkKCiMjU3BhcnNlIE1vZGVsIChyZW1vdmVkIHZhcmlhYmxlcyB0byBicmluZyB0aGUgbnVtYmVyIGJlbG93IDIwKQojc3BhcnNlX21vZGVsID0gbG0obG9nKHByaWNlKSB+IGFyZWEgKyBNUy5ab25pbmcgKyBMb3QuQXJlYSArIAojICAgIE5laWdoYm9yaG9vZCArIENvbmRpdGlvbi4xICsgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgCiMgICAgWWVhci5CdWlsdCArIFllYXIuUmVtb2QuQWRkICsgQnNtdEZpbi5TRi4xICsgQnNtdEZpbi5TRi4yICsgCiMgICAgQnNtdC5VbmYuU0YgKyBDZW50cmFsLkFpciArIEJzbXQuRnVsbC5CYXRoICsgS2l0Y2hlbi5RdWFsICsgCiMgICAgRnVuY3Rpb25hbCArIEdhcmFnZS5DYXJzICsgR2FyYWdlLkFyZWEgKyBoYXMuZmlyZXBsYWNlLCBkYXRhID0gdHJhaW4pCiNzdW1tYXJ5KHNwYXJzZV9tb2RlbCkKI3Bsb3Qoc3BhcnNlX21vZGVsKQojdGVybXBsb3Qoc3BhcnNlX21vZGVsLCAKIyAgICBkYXRhID0gdHJhaW4pCgojI1RyeWluZyB0byBkZXRlY3Qgbm9uLWxpbmVhcml0eQojYXR0YWNoKHRyYWluKQojcGxvdChsb2coYXJlYSksIGxvZyhwcmljZSkpCiNwbG90KGxvZyhMb3QuQXJlYSksIGxvZyhwcmljZSkpCiNwbG90KGxvZyhPdmVyYWxsLlF1YWwpLCBsb2cocHJpY2UpKQojcGxvdChsb2coT3ZlcmFsbC5Db25kKSwgbG9nKHByaWNlKSkKI3Bsb3QobG9nKFllYXIuQnVpbHQpLCBsb2cocHJpY2UpKQojcGxvdChsb2coWWVhci5SZW1vZC5BZGQsIGxvZyhwcmljZSkpKQojcGxvdChCc210RmluLlNGLjEsIGxvZyhwcmljZSkpCiNwbG90KEJzbXRGaW4uU0YuMiwgbG9nKHByaWNlKSkKI3Bsb3QoQnNtdC5VbmYuU0YsIGxvZyhwcmljZSkpCiNwbG90KEJzbXQuRnVsbC5CYXRoLCBsb2cocHJpY2UpKQojcGxvdChHYXJhZ2UuQ2FycywgbG9nKHByaWNlKSkKI3Bsb3QoR2FyYWdlLkFyZWEsIGxvZyhwcmljZSkpCiMKIyNPTEQgU2ltcGxlIG1vZGVsCiNtb2RlbDEgPSBsbShsb2cocHJpY2UpIH4gbG9nKGFyZWEpICsgTVMuWm9uaW5nICsgbG9nKExvdC5BcmVhKSArIAojICAgIE5laWdoYm9yaG9vZCArIENvbmRpdGlvbi4xICsgbG9nKE92ZXJhbGwuUXVhbCkgKyBsb2coT3ZlcmFsbC5Db25kKSArIAojICAgIGxvZyhZZWFyLkJ1aWx0KSArIGxvZyhZZWFyLlJlbW9kLkFkZCkgKyBCc210RmluLlNGLjEgKyBCc210RmluLlNGLjIgKyAKIyAgICBCc210LlVuZi5TRiArIENlbnRyYWwuQWlyICsgQnNtdC5GdWxsLkJhdGggKwojICAgIEZ1bmN0aW9uYWwgKyBHYXJhZ2UuQ2FycyArIEdhcmFnZS5BcmVhICsgaGFzLmZpcmVwbGFjZSwgZGF0YSA9IHRyYWluWy1jKDE2OCwgNDYxLCA3ODcpLF0pCiNzdW1tYXJ5KG1vZGVsMSkKI3Bsb3QobW9kZWwxKQoKI1NpbXBsZSBtb2RlbCAoYmFzZWQgb24gQklDIHNlbGVjdGlvbikKbW9kZWwxID0gbG0oZm9ybXVsYSA9IGxvZyhwcmljZSkgfiBsb2coTG90LkFyZWEpICsgTmVpZ2hib3Job29kICsgQ29uZGl0aW9uLjEgKyAKICAgIE92ZXJhbGwuUXVhbCArIE92ZXJhbGwuQ29uZCArIEhvdXNlQWdlICsgRm91bmRhdGlvbiArIFRvdGFsLkJzbXQuU0YgKyAKICAgIENlbnRyYWwuQWlyICsgbG9nKFgxMi5TRikgKyBCYXRocyArIEtpdGNoZW4uQWJ2R3IgKyBLaXRjaGVuLlF1YWwgKyAKICAgIEZ1bmN0aW9uYWwgKyBGaXJlcGxhY2VzICsgR2FyYWdlLkNhcnMgKyBQYXZlZC5Ecml2ZSArIEJzbXQuWU46QnNtdC5FeHBvc3VyZSwgCiAgICBkYXRhID0gdHJhaW4pCgojUGxvdCBQcmljZSBSYW5nZXMgb2YgZWFjaCBuZWlnaGJvcmhvb2QKbGlicmFyeShnZ3Bsb3QyKQoKbmVpZ2hib3Job29kLnByaWNlLnJhbmdlID0gZ2dwbG90KHRyYWluLCBhZXMoeCA9IGxvZyhwcmljZSksIHkgPSByZW9yZGVyKE5laWdoYm9yaG9vZCwgZGVzYyhsb2cocHJpY2UpKSwgbWVkaWFuKSkpICsKICBnZW9tX3BvbHlnb24oY29sb3I9IlNsYXRlR3JheSIsc2l6ZSA9IDEsIGFscGhhPTEpICsgdGhlbWVfbGlnaHQoKSArIGNvb3JkX2ZsaXAoKSArIHRoZW1lKGF4aXMudGV4dC54PWVsZW1lbnRfdGV4dChhbmdsZT05MCxoanVzdD0xLHZqdXN0PTAuNSkpICsgbGFicyh5PSJOZWlnaGJvcmhvb2QiKQoKcGxvdChuZWlnaGJvcmhvb2QucHJpY2UucmFuZ2UpCmBgYAoKCgojIyMgTW9kZWwgRXZhbHVhdGlvbiBvbiBUZXN0IERhdGEKQ3JlYXRlIHByZWRpY3RlZCB2YWx1ZXMgZm9yIHByaWNlIHVzaW5nIHlvdXIgbW9kZWwgdXNpbmcgdGhlIHRlc3RpbmcgZGF0YQoKYGBge3IgcmVhZC10ZXN0LWRhdGF9CmxvYWQoImFtZXNfdGVzdC5SZGF0YSIpCgojVmFyaWFibGVzIHdpdGggTkEncyBhbmQgdGhlaXIgcHJvcG9ydGlvbiBvZiBtaXNzaW5nIGRhdGEKbWlzcyA9IGFwcGx5KGlzLm5hKGFtZXNfdHJhaW4pLCAyLCBzdW0pCm1pc3NfcHJvcCA9IHJvdW5kKG1pc3NbbWlzcz4wXS9ucm93KGFtZXNfdHJhaW4pLCAzKQpwcmludChtaXNzX3Byb3ApIAp3aGljaChtaXNzX3Byb3A+MC41KSAjIGZvdXIgZmVhdHVyZXMgaGF2ZSBncmVhdGVyIHRoYW4gNTAlIG9mIGRhdGEgbWlzc2luZwoKCiNDcmVhdGVkIGJpbmFyeSB2YXJpYWJsZXMgZm9yIHdoZXRoZXIgb3Igbm90IGEgaG91c2UgYXMgYW4gYWxsZXksIHBvb2wsIGZlbmNlLCBtaXNjLiBmZWF0dXJlLCBmaXJlcGxhY2UsIGJhc2VtZW50IG9yIGdhcmFnZSwgYXMgSSB0aG91Z2h0IGl0IG1heSBiZSBtb3JlIG1lYW5pbmdmdWwtIFRvbQojTWF5IGNvbnNpZGVyIGFkZGluZyBMb3QuRnJvbnRhZ2UgYmFjayBpbiwgYXMgaXQgb25seSBoYWQgMjg2IG1pc3NpbmcKI0hhdmUgdG8gZGVjaWRlIGJldHdlZW4gZmlsdGVyaW5nIG91dCBOL0EncyBmb3IgZGlmZmVyZW50IEdhcmFnZSB2YXJpYWJsZXMgYW5kIG9ubHkgaW5jbHVkaW5nICdoYXMuZ2FyYWdlJwojSGF2ZSB0byBkZWNpZGUgYmV0d2VlbiBmaWx0ZXJpbmcgb3V0IE4vQSdzIGZvciBkaWZmZXJlbnQgQmFzZW1lbnQgdmFyaWFibGVzIGFuZCBvbmx5IGluY2x1ZGluZyAnaGFzLmJhc2VtZW50Jwp0ZXN0IDwtIGFtZXNfdGVzdCAgJT4lICAgZmlsdGVyKCFpcy5uYShMb3QuRnJvbnRhZ2UpKSAlPiUgCiAgbXV0YXRlKEFsbGV5ID0gIGZhY3RvcihBbGxleSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEFsbGV5KSksIGxhYmVscyA9IGMobGV2ZWxzKEFsbGV5KSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoSG91c2VBZ2UgPSBZci5Tb2xkLSBwbWF4KFllYXIuQnVpbHQsIFllYXIuUmVtb2QuQWRkKSkgICU+JQogIGZpbHRlcighaXMubmEoTWFzLlZuci5BcmVhKSkgICU+JSAKICBtdXRhdGUoQnNtdC5ZTiA9ICFpcy5uYShCc210LlF1YWwpKSAlPiUKICAgIG11dGF0ZShCc210LlF1YWwgPSAgZmFjdG9yKEJzbXQuUXVhbCwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEJzbXQuUXVhbCkpLCBsYWJlbHMgPSBjKGxldmVscyhCc210LlF1YWwpLCAiTm9CYSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogICAgbXV0YXRlKEJzbXQuQ29uZCA9ICBmYWN0b3IoQnNtdC5Db25kLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoQnNtdC5Db25kKSksIGxhYmVscyA9IGMobGV2ZWxzKEJzbXQuQ29uZCksICJOb0JhIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgICBtdXRhdGUoQnNtdC5FeHBvc3VyZSA9ICBmYWN0b3IoQnNtdC5FeHBvc3VyZSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEJzbXQuRXhwb3N1cmUpKSwgbGFiZWxzID0gYyhsZXZlbHMoQnNtdC5FeHBvc3VyZSksICJOb0JhIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgIG11dGF0ZShCc210RmluLlR5cGUuMT0gIGZhY3RvcihCc210RmluLlR5cGUuMSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEJzbXRGaW4uVHlwZS4xKSksIGxhYmVscyA9IGMobGV2ZWxzKEJzbXRGaW4uVHlwZS4xKSwgIk5vQmEiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICAgIG11dGF0ZShCc210RmluLlR5cGUuMj0gIGZhY3RvcihCc210RmluLlR5cGUuMiwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEJzbXRGaW4uVHlwZS4yKSksIGxhYmVscyA9IGMobGV2ZWxzKEJzbXRGaW4uVHlwZS4yKSwgIk5vQmEiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoWDEyLlNGPSBYMXN0LkZsci5TRisgWDJuZC5GbHIuU0YpICAlPiUgICAKICBmaWx0ZXIoIWlzLm5hKEJzbXQuRnVsbC5CYXRoKSkgJT4lCiAgZmlsdGVyKCFpcy5uYShCc210LkhhbGYuQmF0aCkpICU+JQogIG11dGF0ZShCYXRocyA9IEJzbXQuRnVsbC5CYXRoICsgMC41KkJzbXQuSGFsZi5CYXRoICsgRnVsbC5CYXRoICsgMC41KkhhbGYuQmF0aCkgJT4lCiAgbXV0YXRlKEZpcmVwbGFjZS5ZTiA9IEZpcmVwbGFjZXM+MCkgJT4lCiAgbXV0YXRlKEZpcmVwbGFjZS5RdSA9ICBmYWN0b3IoRmlyZXBsYWNlLlF1LCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoRmlyZXBsYWNlLlF1KSksIGxhYmVscyA9IGMobGV2ZWxzKEZpcmVwbGFjZS5RdSksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5ZTiA9ICFpcy5uYShHYXJhZ2UuQ29uZCkpICU+JQogICAgbXV0YXRlKEdhcmFnZS5UeXBlID0gIGZhY3RvcihHYXJhZ2UuVHlwZSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEdhcmFnZS5UeXBlKSksIGxhYmVscyA9IGMobGV2ZWxzKEdhcmFnZS5UeXBlKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoR2FyYWdlLkZpbmlzaCA9ICBmYWN0b3IoR2FyYWdlLkZpbmlzaCwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEdhcmFnZS5GaW5pc2gpKSwgbGFiZWxzID0gYyhsZXZlbHMoR2FyYWdlLkZpbmlzaCksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgICBtdXRhdGUoR2FyYWdlLlF1YWwgPSAgZmFjdG9yKEdhcmFnZS5RdWFsLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoR2FyYWdlLlF1YWwpKSwgbGFiZWxzID0gYyhsZXZlbHMoR2FyYWdlLlF1YWwpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogICAgICBtdXRhdGUoR2FyYWdlLkNvbmQgPSAgZmFjdG9yKEdhcmFnZS5Db25kLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoR2FyYWdlLkNvbmQpKSwgbGFiZWxzID0gYyhsZXZlbHMoR2FyYWdlLkNvbmQpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShQb3JjaC5BcmVhID0gV29vZC5EZWNrLlNGKyBPcGVuLlBvcmNoLlNGK0VuY2xvc2VkLlBvcmNoK1gzU3NuLlBvcmNoICsgU2NyZWVuLlBvcmNoKSAlPiUKICBtdXRhdGUoUG9vbC5ZTiA9IFBvb2wuQXJlYT4wKSAlPiUKICBtdXRhdGUoUG9vbC5RQyA9ICBmYWN0b3IoUG9vbC5RQywgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKFBvb2wuUUMpKSwgbGFiZWxzID0gYyhsZXZlbHMoUG9vbC5RQyksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgICBtdXRhdGUoRmVuY2UgPSAgZmFjdG9yKEZlbmNlLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoRmVuY2UpKSwgbGFiZWxzID0gYyhsZXZlbHMoRmVuY2UpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShNaXNjLkZlYXR1cmUgPSAgZmFjdG9yKE1pc2MuRmVhdHVyZSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKE1pc2MuRmVhdHVyZSkpLCBsYWJlbHMgPSBjKGxldmVscyhNaXNjLkZlYXR1cmUpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShNby5Tb2xkID0gYXMuZmFjdG9yKE1vLlNvbGQpKSAlPiUKICAgIG11dGF0ZShZci5Tb2xkID0gYXMuZmFjdG9yKFlyLlNvbGQpKSAlPiUKICBkcGx5cjo6c2VsZWN0KC1HYXJhZ2UuWXIuQmx0KSAlPiUgZmlsdGVyKENvbmRpdGlvbi4xICE9ICJSUk5lIikgJT4lIGZpbHRlcihLaXRjaGVuLlF1YWwgIT0gIlBvIikKICAgICAgICAgICAgICAgICAgCmBgYAoKYGBge3IgcHJlZGljdC1tb2RlbDEsIGVjaG89RkFMU0V9ClloYXQgPSBwcmVkaWN0KG1vZGVsMSwgbmV3ZGF0YT10ZXN0LCBpbnRlcnZhbD0icHJlZGljdGlvbiIpCmBgYAoKWW91IHNob3VsZCBzYXZlIHlvdXIgcHJlZGljdGlvbnMgaW4gYSBkYXRhZnJhbWUgd2l0aCBjb2x1bW5zIGZvciBgUElEYCAgKHByb3BlcnR5IGlkZW50aWZpZXIpLCBgZml0YCwgcHJlZGljdGVkIHZhbHVlcyBvbiB0aGUgdGVzdCBkYXRhLCBhbmQgd2hlcmUgcG9zc2libGUgYGx3cmAgYW5kIGB1cHJgLCBsb3dlciBhbmQgdXBwZXIgOTUlIGludGVydmFsIGVzdGltYXRlcyBmb3IgcHJlZGljdGluZyBgcHJpY2VgLiAKCmBgYHtyIGNyZWF0ZSB9ClkgPSB0ZXN0JHByaWNlClloYXQgPSBleHAoWWhhdCkKCiNCaWFzCm1lYW4oWWhhdFssMV0gLSBZKQoKI01heGltdW0gRGV2aWF0aW9uCm1heChhYnMoWSAtIFloYXRbLDFdKSkKCiNNZWFuIEFic29sdXRlIERldmlhdGlvbgptZWFuKGFicyhZIC0gWWhhdFssMV0pKQoKI1JNU0UKc3FydChtZWFuKChZIC0gWWhhdFssMV0pXjIpKQoKI0NvdmVyYWdlCm1lYW4oWWhhdFssImx3ciJdIDwgWSAmIFloYXRbLCJ1cHIiXSA+IFkpCgojIG5hbWUgZGF0YWZyYW1lIGFzIHByZWRpY3Rpb25zISBETyBOT1QgQ0hBTkdFCnByZWRpY3Rpb25zID0gYXMuZGF0YS5mcmFtZShZaGF0KQpwcmVkaWN0aW9ucyRQSUQgPSB0ZXN0JFBJRApzYXZlKHByZWRpY3Rpb25zLCBmaWxlPSJwcmVkaWN0LlJkYXRhIikKYGBgCgpZb3VyIG1vZGVscyB3aWxsIGJlIGV2YWx1YXRlZCBvbiB0aGUgZm9sbG93aW5nIGNyaXRlcmlhIG9uIHRoZSB0ZXN0IGRhdGE6IAoKKu+CtyBCaWFzOiAgQXZlcmFnZSAoWWhhdC1ZKSAgcG9zaXRpdmUgdmFsdWVzIGluZGljYXRlIHRoZSBtb2RlbCB0ZW5kcyB0byBvdmVyZXN0aW1hdGUgcHJpY2UgKG9uIGF2ZXJhZ2UpIHdoaWxlIG5lZ2F0aXZlIHZhbHVlcyBpbmRpY2F0ZSB0aGUgbW9kZWwgdGVuZHMgdG8gdW5kZXJlc3RpbWF0ZSBwcmljZS4KCirvgrcgTWF4aW11bSBEZXZpYXRpb246ICBNYXggfFktWWhhdHwgLSAgaWRlbnRpZmllcyB0aGUgd29yc3QgcHJlZGljdGlvbiAgbWFkZSBpbiB0aGUgdmFsaWRhdGlvbiBkYXRhIHNldC4KCirvgrcgTWVhbiBBYnNvbHV0ZSBEZXZpYXRpb246ICBBdmVyYWdlIHxZLVloYXR8IC0gdGhlIGF2ZXJhZ2UgZXJyb3IgKHJlZ2FyZGxlc3Mgb2Ygc2lnbikuCgoqIFJvb3TvgrcgTWVhbiBTcXVhcmUgRXJyb3I6IFNxcnQgQXZlcmFnZSAoWS1ZaGF0KV4yCgoqIENvdmVyYWdlOiAgQXZlcmFnZSggbHdyIDwgWSA8IHVwcikgCgpJbiBvcmRlciB0byBoYXZlIGEgcGFzc2luZyB3ZXJja2VyIGJhZGdlLCB5b3VyIGZpbGUgZm9yIHByZWRpY3Rpb25zIG5lZWRzIHRvIGJlIHRoZSBzYW1lIGxlbmd0aCBhcyB0aGUgdGVzdCBkYXRhLCB3aXRoIHRocmVlIGNvbHVtbnM6ICBmaXR0ZWQgdmFsdWVzLCBsb3dlciBDSSBhbmQgdXBwZXIgQ0kgdmFsdWVzIGluIHRoYXQgb3JkZXIgd2l0aCBuYW1lcywgZml0LCBsd3IsIGFuZCB1cHIgcmVzcGVjdGl2ZWx5LgoKWW91IHdpbGwgYmUgYWJsZSB0byBzZWUgeW91ciBzY29yZXMgb24gdGhlIHNjb3JlIGJvYXJkIChjb21pbmcgc29vbiEpLiAgVGhleSB3aWxsIGJlIGluaXRpYWxpemVkIGJ5IGEgcHJlZGN0aW9uIGJhc2VkIG9uIHRoZSBtZWFuIGluIHRoZSB0cmFpbmluZyBkYXRhLgoKX01vZGVsIENoZWNrXyAtIFRlc3QgeW91ciBwcmVkaWN0aW9uIG9uIHRoZSBmaXJzdCBvYnNlcnZhdGlvbiBpbiB0aGUgdHJhaW5pbmcgYW5kIHRlc3QgZGF0YSBzZXQgdG8gbWFrZSBzdXJlIHRoYXQgdGhlIG1vZGVsIGdpdmVzIGEgcmVhc29uYWJsZSBhbnN3ZXIgYW5kIGluY2x1ZGUgdGhpcyBpbiBhIHN1cHBsZW1lbnQgb2YgeW91ciByZXBvcnQuIFRoaXMgc2hvdWxkIGJlIGRvbmUgQlkgSEFORCB1c2luZyBhIGNhbGN1bGF0b3IgKHRoaXMgbWVhbnMgdXNlIHRoZSByYXcgZGF0YSBmcm9tIHRoZSBvcmlnaW5hbCBkYXRhc2V0IGFuZCBtYW51YWxseSBjYWxjdWxhdGUgYWxsIHRyYW5zZm9ybWF0aW9ucyBhbmQgaW50ZXJhY3Rpb25zIHdpdGggeW91ciBjYWxjdWxhdG9yKSEgTW9kZWxzIHRoYXQgZG8gbm90IGdpdmUgcmVhc29uYWJsZSBhbnN3ZXJzIHdpbGwgYmUgZ2l2ZW4gYSBtaW5pbXVtIDIgbGV0dGVyIGdyYWRlIHJlZHVjdGlvbi4gQWxzbyBiZSBjYXJlZnVsIGFzIHlvdSBjYW5ub3QgdXNlIGNlcnRhaW4gdHJhbnNmb3JtYXRpb25zIFtsb2cgb3IgaW52ZXJzZSB4XSBpZiBhIHZhcmlhYmxlIGhhcyB2YWx1ZXMgb2YgMC4KCgoKCiMjIyBQYXJ0IElJOiBDb21wbGV4IE1vZGVsCgpJbiB0aGlzIHBhcnQgeW91IG1heSBnbyBhbGwgb3V0IGZvciBjb25zdHJ1Y3RpbmcgYSBiZXN0IGZpdHRpbmcgbW9kZWwgZm9yIHByZWRpY3RpbmcgaG91c2luZyBwcmljZXMgdXNpbmcgbWV0aG9kcyB0aGF0IHdlIGhhdmUgY292ZXJlZCB0aGlzIHNlbWVzdGVyLiAgWW91IHNob3VsZCBmZWVsIGZyZWUgdG8gdG8gY3JlYXRlIGFueSBuZXcgdmFyaWFibGVzIChzdWNoIGFzIHF1YWRyYXRpYywgaW50ZXJhY3Rpb24sIG9yIGluZGljYXRvciB2YXJpYWJsZXMsIHNwbGluZXMsIGV0YykuIFRoZSB2YXJpYWJsZSBgVG90YWxTcSA9IFgxc3QuRmxyLlNGK1gybmQuRmxyLlNGYCB3YXMgYWRkZWQgdG8gdGhlIGRhdGFmcmFtZSAodGhhdCBkb2VzIG5vdCBpbmNsdWRlIGJhc2VtZW50IGFyZWEsIHNvIHlvdSBtYXkgaW1wcm92ZSBvbiB0aGlzLiBBIHJlbGF0aXZlIGdyYWRlIGlzIGFzc2lnbmVkIGJ5IGNvbXBhcmluZyB5b3VyIGZpdCBvbiB0aGUgdGVzdCBzZXQgdG8gdGhhdCBvZiB5b3VyIGZlbGxvdyBzdHVkZW50cyB3aXRoIGJvbnVzIHBvaW50cyBhd2FyZGVkIHRvIHRob3NlIHdobyBzdWJzdGFudGlhbGx5IGV4Y2VlZCB0aGVpciBmZWxsb3cgc3R1ZGVudHMgYW5kIHBvaW50IHJlZHVjdGlvbnMgb2NjdXJyaW5nIGZvciBtb2RlbHMgd2hpY2ggZml0IGV4Y2VlZGluZ2x5IHBvb3JseS4gIAoKYGBge3J9CmxpYnJhcnkoZ2xtbmV0KQojRGVzaWduIG1hdHJpY2VzClgudHJhaW4gPSBtb2RlbC5tYXRyaXgobG9nKHByaWNlKSB+IGxvZyhMb3QuQXJlYSkgKyBOZWlnaGJvcmhvb2QgKyBDb25kaXRpb24uMSArIAogICAgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgSG91c2VBZ2UgKyBGb3VuZGF0aW9uICsgVG90YWwuQnNtdC5TRiArIAogICAgQ2VudHJhbC5BaXIgKyBsb2coWDEyLlNGKSArIEJhdGhzICsgS2l0Y2hlbi5BYnZHciArIEtpdGNoZW4uUXVhbCArIAogICAgRnVuY3Rpb25hbCArIEZpcmVwbGFjZXMgKyBHYXJhZ2UuQ2FycyArIFBhdmVkLkRyaXZlICsgQnNtdC5ZTjpCc210LkV4cG9zdXJlLCAKICAgIGRhdGEgPSBkYXRhKQoKWC50ZXN0ID0gbW9kZWwubWF0cml4KGxvZyhwcmljZSkgfiBsb2coTG90LkFyZWEpICsgTmVpZ2hib3Job29kICsgQ29uZGl0aW9uLjEgKyAKICAgIE92ZXJhbGwuUXVhbCArIE92ZXJhbGwuQ29uZCArIEhvdXNlQWdlICsgRm91bmRhdGlvbiArIFRvdGFsLkJzbXQuU0YgKyAKICAgIENlbnRyYWwuQWlyICsgbG9nKFgxMi5TRikgKyBCYXRocyArIEtpdGNoZW4uQWJ2R3IgKyBLaXRjaGVuLlF1YWwgKyAKICAgIEZ1bmN0aW9uYWwgKyBGaXJlcGxhY2VzICsgR2FyYWdlLkNhcnMgKyBQYXZlZC5Ecml2ZSArIEJzbXQuWU46QnNtdC5FeHBvc3VyZSwgCiAgICBkYXRhID0gdGVzdCkKCiNGaXQgbGFzc28gcmVncmVzc2lvbgpwcmljZS5sYXNzbyA9IGdsbW5ldChYLnRyYWluLCBsb2coZGF0YSRwcmljZSksIGFscGhhPTEpCgpjdi5sYXNzbyA9IGN2LmdsbW5ldChYLnRyYWluLCBsb2coZGF0YSRwcmljZSksIGFscGhhPTEpCgojT2J0YWluIHByZWRpY3Rpb24gb24gdGVzdCBkYXRhCnloYXQubGFzc28udGVzdCA9IHByZWRpY3QoYXBwLmxhc3NvLCBzPWN2Lmxhc3NvJGxhbWJkYS5taW4sIHR5cGU9InJlc3BvbnNlIiwgbmV3eCA9IFgudGVzdCkKCiNDb21wdXRlIFJNU0UgZm9yIHRlc3QgZGF0YQpybXNlLmxhc3NvLnRlc3QgPSBybXNlKHRlc3QkcHJpY2UsIGV4cCh5aGF0Lmxhc3NvLnRlc3QpKQpybXNlLnRhYiRsYXNzbzwtcm1zZS5sYXNzby50ZXN0CmBgYAoKVXBkYXRlIHlvdXIgcHJlZGljdGlvbnMgdXNpbmcgeW91ciBjb21wbGV4IG1vZGVsIHRvIHByb3ZpZGUgcG9pbnQgZXN0aW1hdGVzIGFuZCBDSS4KCmBgYHtyIHByZWRpY3QtbW9kZWwyLCBlY2hvPUZBTFNFfQojIHJlcGxhY2UgbW9kZWwxIHdpdGggbW9kZWwyCnByZWRpY3Rpb25zID0gYXMuZGF0YS5mcmFtZShwcmVkaWN0KG1vZGVsMSwgbmV3ZGF0YT1hbWVzX3Rlc3QsIGludGVydmFsID0gInByZWQiKSkKcHJlZGljdGlvbnMkUElEID0gYW1lc190ZXN0JFBJRApzYXZlKHByZWRpY3Rpb25zLCBmaWxlPSJwcmVkaWN0LlJkYXRhIikKYGBgCgpZb3UgbWF5IGl0ZXJhdGUgaGVyZSBhcyBtdWNoIGFzIHlvdSBsaWtlIGV4cGxvcmluZyBkaWZmZXJlbnQgbW9kZWxzIHVudGlsIHlvdSBhcmUgc2F0aXNmaWVkIHdpdGggeW91ciByZXN1bHRzLgoKCgoKQmF5ZXNpYW4gbGFzc28KYGBge3J9CmxpYnJhcnkoZ2xtbmV0KQpsaWJyYXJ5KG1vbm9tdm4pCgpob3VzaW5nLnJpZGdlID0gbG0ucmlkZ2UobG9nKHByaWNlKSB+IChsb2coTG90LkFyZWEpICsgTmVpZ2hib3Job29kICsgQ29uZGl0aW9uLjEgKyAKICAgIE92ZXJhbGwuUXVhbCArIE92ZXJhbGwuQ29uZCArIEhvdXNlQWdlICsgRm91bmRhdGlvbiArIFRvdGFsLkJzbXQuU0YgKyAKICAgIENlbnRyYWwuQWlyICsgbG9nKFgxMi5TRikgKyBCYXRocyArIEtpdGNoZW4uQWJ2R3IgKyBLaXRjaGVuLlF1YWwgKyAKICAgIEZ1bmN0aW9uYWwgKyBGaXJlcGxhY2VzICsgR2FyYWdlLkNhcnMgKyBQYXZlZC5Ecml2ZSleMiArIEJzbXQuWU46QnNtdC5FeHBvc3VyZSwgCiAgICAgICAgICAgICAgICAgICAgICAgICBsYW1iZGEgPSBzZXEoMCwgNSwgMC4wMDAxKSwgZGF0YSA9IGRhdGEpIAoKZGF0YTIgPSBkYXRhICU+JSBtdXRhdGUoYnNtdC5xdWFsID0gQnNtdC5ZTipCc210LlF1YWwsIGJzbXQuY29uZCA9IEJzbXQuWU46QnNtdC5Db25kLAogICAgICAgICAgICAgICAgICAgICAgICBic210LmV4cG9zdXJlID0gQnNtdC5ZTjpCc210LkV4cG9zdXJlKQoKbG9hZCgifi9EZXNrdG9wL1NUQSA1MjEvUHJvamVjdF9ob21vQmF5ZXNpYW5zL2FtZXNfdGVzdC5SZGF0YSIpCmN0ZXN0ID0gYW1lc190ZXN0CmN0cmFpbiA9IGRhdGEKCiMgcmlkZ2U6CmhvdXNpbmcucmlkZ2UgPSBsbS5yaWRnZShsb2cocHJpY2UpIH4gKE1TLlN1YkNsYXNzICsgTVMuWm9uaW5nKyBsb2coTG90LkZyb250YWdlKSArIGxvZyhMb3QuQXJlYSkgKyBTdHJlZXQpXjIsCiAgICAgICAgICAgICAgICAgICAgICAgICBsYW1iZGEgPSBzZXEoMCwgNSwgMC4wMDAxKSwgZGF0YSA9IGRhdGEpIAoKYmVzdC5sYW1iZGEgPSBob3VzaW5nLnJpZGdlJGxhbWJkYVt3aGljaC5taW4oaG91c2luZy5yaWRnZSRHQ1YpXQoKCgojIGxhc3NvOgojIFggYW5kIFk6ClggPSBtb2RlbC5tYXRyaXgobG9nKHByaWNlKX4gKGxvZyhMb3QuQXJlYSkgKyBOZWlnaGJvcmhvb2QgKyBDb25kaXRpb24uMSArIAogICAgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgSG91c2VBZ2UgKyBGb3VuZGF0aW9uICsgVG90YWwuQnNtdC5TRiArIAogICAgQ2VudHJhbC5BaXIgKyBsb2coWDEyLlNGKSArIEJhdGhzICsgS2l0Y2hlbi5BYnZHciArIEtpdGNoZW4uUXVhbCArIAogICAgRnVuY3Rpb25hbCArIEZpcmVwbGFjZXMgKyBHYXJhZ2UuQ2FycyArIFBhdmVkLkRyaXZlKV4yICsgQnNtdC5ZTjpCc210LkV4cG9zdXJlLCAKICAgICAgICAgICAgICAgICBkYXRhID0gZGF0YSkKWSA9IGFzLm1hdHJpeChkYXRhWywzXSkKCiMgbGFzc286Cmxhc3NvLmN2ID0gY3YuZ2xtbmV0KHggPSBYLCB5ID0gWSwgYWxwaGE9MSwgbGFtYmRhID0gc2VxKDAsIDIsIDAuMDAxKSwgZmFtaWx5ID0gImdhdXNzaWFuIikKbGFtYmRhLmxhc3NvID0gbGFzc28uY3YkbGFtYmRhLm1pbgojIGxhc3NvIHdpdGggdGhlIGJlc3QgbGFtYmRhCmhvdXNpbmcubGFzc28gPSBnbG1uZXQoWCwgWSwgYWxwaGE9MSwgbGFtYmRhID0gbGFtYmRhLmxhc3NvLCBmYW1pbHkgPSAiZ2F1c3NpYW4iKSAKY29lZihob3VzaW5nLmxhc3NvKQpgYGAKCgpgYGB7cn0KbGlicmFyeShtb25vbXZuKQojU0NBTEVEPz8KCgpYID0gbW9kZWwubWF0cml4KGxvZyhwcmljZSl+IGxvZyhMb3QuQXJlYSkgKyBhcy5udW1lcmljKE5laWdoYm9yaG9vZCkgKyBhcy5udW1lcmljKENvbmRpdGlvbi4xKSArIAogICAgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgSG91c2VBZ2UgKyBhcy5udW1lcmljKEZvdW5kYXRpb24pICsgVG90YWwuQnNtdC5TRiArIAogICAgYXMubnVtZXJpYyhDZW50cmFsLkFpcikgKyBsb2coWDEyLlNGKSArIEJhdGhzICsgS2l0Y2hlbi5BYnZHciArIGFzLm51bWVyaWMoS2l0Y2hlbi5RdWFsKSArIAogICAgYXMubnVtZXJpYyhGdW5jdGlvbmFsKSArIEZpcmVwbGFjZXMgKyBHYXJhZ2UuQ2FycyArIGFzLm51bWVyaWMoUGF2ZWQuRHJpdmUpXjIgKyBhcy5udW1lcmljKEJzbXQuWU4pKmFzLm51bWVyaWMoQnNtdC5FeHBvc3VyZSksIAogICAgICAgICAgICAgICAgIGRhdGEgPSBkYXRhKQoKWC5zY2FsZWQgPSBzY2FsZSh1bmxpc3QoWCksIGNlbnRlciA9IFRSVUUsIHNjYWxlID0gVFJVRSkKWSA9IHVubGlzdChkYXRhWywgM10pCgpob3VzaW5nLmJsYXNzby5SSiA9IGJsYXNzbyhYLnNjYWxlZCwgWSwgUkogPSBUUlVFLCB2ZXJiPTApCnlwcmVkLmJsYXNzby5SSiA9IG1lYW4oaG91c2luZy5ibGFzc28uUkokbXUpICsgYXMubWF0cml4KHRlc3Quc2NhbGVkKSAlKiUgYXBwbHkoaG91c2luZy5ibGFzc28uUkokYmV0YSwgMiwgbWVhbikKc3VtbWFyeShob3VzaW5nLmJsYXNzby5SSikkY29lZgoKIyBwcmVkaWN0aW9uOgojIHlwcmVkLmJsYXNzby5SSiA9IG1lYW4oY29sbGVnZS5ibGFzc28uUkokbXUpICsgYXMubWF0cml4KHRlc3Quc2NhbGVkKSAlKiUgYXBwbHkoY29sbGVnZS5ibGFzc28uUkokYmV0YSwgMiwgbWVhbikKIyBybXNlLmJsYXNzby5SSiA9IHJtc2UoZXhwKGN0ZXN0JEFwcHMpLCBleHAoeXByZWQuYmxhc3NvLlJKKSkgCmBgYAoKCmBgYHtyfQojIGNlbnRlciBhbmQgc2NhbGUgWApYLnNjYWxlZCA9IFgKWC5zY2FsZWRbLDI6MTVdID0gc2NhbGUoWFssMjoxNV0sIGNlbnRlciA9Y29sbGVnZS5yaWRnZS5jdiR4bVtjKC0xLC0xNildLAogICAgICAgICAgICAgICAgICAgICAgICBzY2FsZSA9IGNvbGxlZ2UucmlkZ2UuY3Ykc2NhbGVzW2MoLTEsLTE2KV0pCiAgICAgICAgICAgICAgICAgICAgICAgIAojIFJKID0gRkFMU0UKIyBjb2xsZWdlLmJsYXNzbyA9IGJsYXNzbyhYLnNjYWxlZCwgWSwgUkogPSBGQUxTRSwgdmVyYj0wKSAKIyB5cHJlZC5ibGFzc28gPSBtZWFuKGNvbGxlZ2UuYmxhc3NvJG11KSArCiMgICAgICAgICAgICAgICAgYXMubWF0cml4KHRlc3Quc2NhbGVkKSAlKiUgYXBwbHkoY29sbGVnZS5ibGFzc28kYmV0YSwgMiwgbWVhbikgCiMgcm1zZS5ibGFzc28gPSBybXNlKGV4cChjdGVzdCRBcHBzKSwgZXhwKHlwcmVkLmJsYXNzbykpCiMgbW9kZWwgc2VsZWN0aW9uIHdpdGggUkogPSBUUlVFCmNvbGxlZ2UuYmxhc3NvLlJKID0gYmxhc3NvKFguc2NhbGVkLCBZLCBSSiA9IFRSVUUsIHZlcmI9MCkKeXByZWQuYmxhc3NvLlJKID0gbWVhbihjb2xsZWdlLmJsYXNzby5SSiRtdSkgKyBhcy5tYXRyaXgodGVzdC5zY2FsZWQpICUqJSBhcHBseShjb2xsZWdlLmJsYXNzby5SSiRiZXRhLCAyLCBtZWFuKQpybXNlLmJsYXNzby5SSiA9IHJtc2UoZXhwKGN0ZXN0JEFwcHMpLCBleHAoeXByZWQuYmxhc3NvLlJKKSkgCnN1bW1hcnkoY29sbGVnZS5ibGFzc28uUkopJGNvZWYKYGBgCgoKYGBge3J9Cm1vZGVsLmxhc3NvID0gZ2xtbmV0KFgudHJhaW4sIGxvZyhkYXRhX3RyYWluJHByaWNlKSwgYWxwaGE9MSkKY3YubGFzc28gPSBjdi5nbG1uZXQoWC50cmFpbiwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpLCBhbHBoYT0xKQp5aGF0Lmxhc3NvID0gZXhwKHByZWRpY3QobW9kZWwubGFzc28sIHM9Y3YubGFzc28kbGFtYmRhLm1pbiwgdHlwZT0icmVzcG9uc2UiLCBuZXd4ID0gWC50ZXN0KSkKCnNxcnQobWVhbigoKHloYXQubGFzc28pLWRhdGFfdGVzdCRwcmljZSleMikpCgoKIyBSaWRnZSBmb3IgdGhlIHNpbXBsaWZpZWQgbW9kZWwgKGZyb20gQklDKQoKbW9kZWwucmlkZ2UgPSBnbG1uZXQoWC50cmFpbiwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpLCBhbHBoYT0wKQpjdi5yaWRnZSA9IGN2LmdsbW5ldChYLnRyYWluLCBsb2coZGF0YV90cmFpbiRwcmljZSksIGFscGhhPTApCnloYXQucmlkZ2UgPSBwcmVkaWN0KG1vZGVsLnJpZGdlLCBzPWN2LnJpZGdlJGxhbWJkYS5taW4sIHR5cGU9InJlc3BvbnNlIiwgbmV3eCA9IFgudGVzdCkKCnNxcnQobWVhbigoZXhwKHloYXQucmlkZ2UpLWRhdGFfdGVzdCRwcmljZSleMikpCmBgYAoKCgoKCgoKCiMjIyBQYXJ0IElJSTogV3JpdGUgVXAKCk9uY2UgeW91IGFyZSBzYXRpc2ZpZWQgd2l0aCB5b3VyIG1vZGVsLCBwcm92aWRlIGEgd3JpdGUgdXAgb2YgeW91ciBkYXRhIGFuYWx5c2lzIHByb2plY3QgaW4gYSBuZXcgUm1kIGZpbGUvcGRmIGZpbGU6IGB3cml0ZXVwLlJtZGAgYnkgY29weWluZyBvdmVyIHNhbGllbnQgcGFydHMgb2YgeW91ciBSIG5vdGVib29rLiBUaGUgd3JpdHRlbiBhc3NpZ25tZW50IGNvbnNpc3RzIG9mIGZpdmUgcGFydHM6CgoxLiBFeHBsb3JhdG9yeSBkYXRhIGFuYWx5c2lzICgyMCBwb2ludHMpOiBtdXN0IGluY2x1ZGUgdGhyZWUgY29ycmVjdGx5IGxhYmVsZWQgZ3JhcGhzIGFuZCBhbiBleHBsYW5hdGlvbiB0aGF0IGhpZ2hsaWdodCB0aGUgbW9zdCBpbXBvcnRhbnQgZmVhdHVyZXMgdGhhdCB3ZW50IGludG8geW91ciBtb2RlbCBidWlsZGluZy4KCgoyLiBEZXZlbG9wbWVudCBhbmQgYXNzZXNzbWVudCBvZiBhbiBpbml0aWFsIG1vZGVsIGZyb20gUGFydCBJICgxMCBwb2ludHMpCgoqIEluaXRpYWwgbW9kZWw6IG11c3QgaW5jbHVkZSBhIHN1bW1hcnkgdGFibGUgYW5kIGFuIGV4cGxhbmF0aW9uL2Rpc2N1c3Npb24gZm9yIHZhcmlhYmxlIHNlbGVjdGlvbi4gIEludGVycHJldGF0aW9uIG9mIGNvZWZmaWNpZW50cyBkZXNpcmFibGUgZm9yIGZ1bGwgcG9pbnRzLgoKKiBNb2RlbCBzZWxlY3Rpb246IG11c3QgaW5jbHVkZSBhIGRpc2N1c3Npb24KCiogUmVzaWR1YWw6IG11c3QgaW5jbHVkZSBhIHJlc2lkdWFsIHBsb3QgYW5kIGEgZGlzY3Vzc2lvbgoKKiBSTVNFOiBtdXN0IGluY2x1ZGUgYW4gUk1TRSBhbmQgYW4gZXhwbGFuYXRpb24gIChvdGhlciBjcml0ZXJpYSBkZXNpcmFibGUpCgoqIE1vZGVsIHRlc3Rpbmc6IG11c3QgaW5jbHVkZSBhbiBleHBsYW5hdGlvbgoKCgozLiBEZXZlbG9wbWVudCBvZiB0aGUgZmluYWwgbW9kZWwgKDIwIHBvaW50cykKCiogRmluYWwgbW9kZWw6IG11c3QgaW5jbHVkZSBhIHN1bW1hcnkgdGFibGUKCiogVmFyaWFibGVzOiBtdXN0IGluY2x1ZGUgYW4gZXhwbGFuYXRpb24KCiogVmFyaWFibGUgc2VsZWN0aW9uL3Nocmlua2FnZTogbXVzdCB1c2UgYXBwcm9wcmlhdGUgbWV0aG9kIGFuZCBpbmNsdWRlIGFuIGV4cGxhbmF0aW9uCgoKCjQuIEFzc2Vzc21lbnQgb2YgdGhlIGZpbmFsIG1vZGVsICgyNSBwb2ludHMpCgoqIFJlc2lkdWFsOiBtdXN0IGluY2x1ZGUgYSByZXNpZHVhbCBwbG90IGFuZCBhIGRpc2N1c3Npb24KCiogUk1TRTogbXVzdCBpbmNsdWRlIGFuIFJNU0UgYW5kIGFuIGV4cGxhbmF0aW9uICAob3RoZXIgY3JpdGVyaWEgZGVzaXJhYmxlKQoKKiBNb2RlbCBldmFsdWF0aW9uOiBtdXN0IGluY2x1ZGUgYW4gZXZhbHVhdGlvbiBkaXNjdXNzaW9uCgoqIE1vZGVsIHRlc3RpbmcgOiBtdXN0IGluY2x1ZGUgYSBkaXNjdXNzaW9uCgoqIE1vZGVsIHJlc3VsdDogbXVzdCBpbmNsdWRlIGEgc2VsZWN0aW9uIG9mIHRoZSB0b3AgMTAgdW5kZXJ2YWx1ZWQgYW5kIG92ZXJ2YWx1ZWQgaG91c2VzCgoKCjUuIENvbmNsdXNpb24gKDEwIHBvaW50cyk6IG11c3QgaW5jbHVkZSBhIHN1bW1hcnkgb2YgcmVzdWx0cyBhbmQgYSBkaXNjdXNzaW9uIG9mIHRoaW5ncyBsZWFybmVkCgoKCiMjIyBQYXJ0IElWCkNyZWF0ZSBwcmVkaWN0aW9ucyBmb3IgdGhlIHZhbGlkYXRpb24gZGF0YSBmcm9tIHlvdXIgZmluYWwgbW9kZWwgYW5kIHdyaXRlIG91dCB0byBhIGZpbGUgYHByZWRpY3Rpb24tdmFsaWRhdGlvbi5SZGF0YWAKVGhpcyBzaG91bGQgaGF2ZSB0aGUgc2FtZSBmb3JtYXQgYXMgdGhlIG1vZGVscyBpbiBQYXJ0IEkgYW5kIElJLgoKMTAgcG9pbnRzCgoKCiMjIyBDbGFzcyBQcmVzZW50YXRpb25zCgpFYWNoIEdyb3VwIHNob3VsZCBwcmVwYXJlIDUgc2xpZGVzIGluIHRoZWlyIEdpdGh1YiByZXBvOiAgKHNhdmUgYXMgc2xpZGVzLnBkZikKCiogTW9zdCBpbnRlcmVzdGluZyBncmFwaGljICAoYSBwaWN0dXJlIGlzIHdvcnRoIGEgdGhvdXNhbmQgd29yZHMgcHJpemUhKSAgCgoqIEJlc3QgTW9kZWwgKG1vdGl2YXRpb24sIGhvdyB5b3UgZm91bmQgaXQsIHdoeSB5b3UgdGhpbmsgaXQgaXMgYmVzdCkKCiogQmVzdCBJbnNpZ2h0cyBpbnRvIHByZWRpY3RpbmcgU2FsZXMgUHJpY2UuCgoqIDIgQmVzdCBIb3VzZXMgdG8gcHVyY2hhc2UgIChhbmQgd2h5KQoKKiBCZXN0IFRlYW0gTmFtZS9HcmFwaGljCgpXZSB3aWxsIHNlbGVjdCB3aW5uZXJzIGJhc2VkIG9uIHRoZSBhYm92ZSBjcml0ZXJpYSBhbmQgb3ZlcmFsbCBwZXJmb3JtYW5jZS4KCgpGaW5hbGx5IHlvdXIgcmVwbyBzaG91bGQgaGF2ZTogYHdyaXRldXAuUm1kYCwgYHdyaXRldXAucGRmYCwgYHNsaWRlcy5SbWRgIChhbmQgd2hhdGV2ZXIgb3V0cHV0IHlvdSB1c2UgZm9yIHRoZSBwcmVzZW50YXRpb24pIGFuZCBgcHJlZGljdC5SZGF0YWAgYW5kIGBwcmVkaWN0LXZhbGlkYXRpb24uUmRhdGFgLg==